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Abstract 

This  research  explored  the  use  of  modeling  and  enterprise  architecture  in  the 
analysis  of  Air  Force  Capabilities.  The  Air  Force  accomplishes  this  through  the 
Capability  Review  and  Risk  Assessment  (CRRA).  The  CRRA  is  currently  performed  by 
building  architectures  which  contain  Process  Sequence  Models  (PSMs).  PSMs  are  scored 
by  Subject  Matter  Experts  to  determine  the  probability  of  successfully  completing  the 
mission  they  model  and  ultimately  to  determine  the  risk  associated  to  Air  Force 
capabilities.  Two  findings  were  identified.  The  first  is  that  creating  additional 
architectural  viewpoints,  some  of  which  are  currently  being  proposed  for  version  2.0  of 
the  DoD  Architecture  Framework,  can  benefit  CRRA  development.  The  second  is  PSMs 
have  fundamental  limitations  associated  with  the  inability  to  capture  dependencies  among 
activities  as  well  as  the  inability  to  get  beyond  binary  success  criteria  to  address  issues  of 
capability  sufficiency.  To  remedy  these  limitations  a  model  called  Extended  Sequence 
Models  (ESMs)  was  developed.  ESMs  extend  PSMs  by  using  reliability  modeling 
techniques  combined  with  linear  regression  to  show  dependencies  between  components. 
This  model  also  allows  the  effects  of  capability  sufficiency  to  be  captured  and  related  to 
mission  success. 
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APPLICATION  OF  RELIABILITY  AND  LINEAR  REGRESSION  TO  ENTERPRISE 


ARCHITECTURE  IN  SUPPORT  OF  THE  US  AIR  FORCE’S  CAPABILITY  REVIEW 

AND  RISK  ASSESSMENT 


1.  Introduction 

In  2003  the  Department  of  Defense  (DoD)  changed  the  method  it  uses  to  define 
operational  needs  by  transitioning  from  the  Requirements  Generation  System  to  the  Joint 
Capabilities  Integration  and  Development  System  (JCIDS).  This  change  was  captured  in 
the  Chairman  of  the  Joint  Chief  of  Staff  Instruction  (CJCSI)  3170.01  series.  The  DoD 
Joint  Staff  described  this  change  as  one  from  a  service-focused,  threat-based  analysis  to 
one  that  is  Joint-focused  and  Capability-based  (JCS,  2005:6).  Capability,  as  used  in  this 
research,  is  “the  ability  to  achieve  a  desired  effect  under  specified  standards  and 
conditions  through  combinations  of  means  and  ways  to  perform  a  set  of  tasks”  as  defined 
in  the  CJCSI  3 170.0 IF  (JCIDS).  JCIDS  is  a  key  supporting  process  to  the  DoD 
Acquisition,  Programming,  and  Budgeting  processes.  JCIDS  includes  three  main 
analyses:  identify  the  capabilities  needed  to  perform  required  missions  (Functional  Area 
Analysis,  or  FAA),  evaluate  the  current  force’s  ability  to  meet  those  capabilities 
(Functional  Needs  Analysis,  or  FNA),  and  identify  possible  solutions  to  eliminate  any 
capability  shortfalls  (Functional  Solutions  Analysis,  or  FSA).  In  the  language  often  used 
within  the  DoD,  the  purpose  of  this  process  is  to  “fill  the  capability  gap”  (CJCS, 

2007:  A5).  The  Air  Force  accomplishes  these  analyses  for  the  capabilities  it  is  expected 
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to  deliver  to  the  joint  force  with  Called  Capability  Based  Planning  (CBP)  as  described  in 
Air  Force  Instructions  (AFI)  10-601  Capability  Based  Requirement  Development  and 
AFI  10-604  Capability  Based  Planning.  Within  this  larger  planning  method,  the  Air 
Force  uses  a  sub-process  to  identify  its  needed  capabilities  and  determine  its  ability  to 
deliver  those  capabilities,  the  first  two  analyses  of  the  JCIDS  process.  The  Air  Force 
accomplishes  this  through  a  risk  analysis  method  called  the  Capabilities  Review  and  Risk 
Assessment  (CRRA)  (DAF,  2006:  6).  Additionally,  the  analytic  models  used  to 
accomplish  the  CRRA  are  Process  Sequence  Models  (PSMs). 

Years  before  the  DoD  made  its  transition  to  capability  based  planning,  many  in 
the  federal  government  recognized  a  need  to  better  document  the  architecture  of  the 
systems  that  were  developed  and  used.  In  this  use,  architecture  is  “the  structure  of 
components,  their  relationships,  and  the  principles  and  guidelines  governing  their  design 
and  evolution  over  time”  (DoD,  2007:  ES-1).  This  realization  rose  from  the  information 
technology  community’s  efforts  to  ensure  system  interoperability.  Benefits  of  building 
system  architectures  were  deemed  important  enough  that  statutory  requirements  were 
developed  to  mandate  their  construction  in  various  situations.  A  few  examples  of  these 
requirements  are  the  Clinger-Cohen  Act  of  1996,  Office  of  Management  and  Budget 
Circular  A-130,  the  E-Govemment  Act  of  2002  (DoD,  2007:  3-2);  and  even  JCIDS. 

Over  the  course  of  the  last  15  years,  there  has  been  a  growing  appreciation  for  the 
benefits  of  documenting  the  architecture  of  various  types  of  systems  such  as 
manufacturing  systems,  social  systems,  and  political  systems  to  name  a  few  (Maier  and 
Rechtin,  2002). 
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1.1  Research  Questions 

This  research  answers  several  questions.  The  first  is,  ‘How  is  architecture 
currently  being  used  to  support  the  CRRA?’  Included  is  an  examination  of  PSMs  to 
include  their  assumptions  and  limitations.  The  second  question  is,  ‘How  can  the  current 
architectural  models  be  extended  to  make  them  more  appropriate  and  useful?’  The  third 
question  is,  ‘What  other  architectural  models  can  be  built  and  analyzed  that  would  assist 
the  CRRA?’ 

Since  the  CRRA  is  a  sub-process  of  the  larger  CBP  process  the  boundaries 
between  the  two  can  be  somewhat  ambiguous.  This  study  considers  the  CRRA  to 
encompass  the  activities  needed  to  identify  Air  Force  capability  shortfalls,  gaps,  and  trade 
space  as  described  in  AFI 10-604.  For  the  purpose  of  this  study  it  does  not  include  the 
supporting  processes  of  developing  and  writing  Air  Force  Concept  of  Operations 
(CONOPS)  which  are  covered  in  Air  Force  Policy  Document  (AFPD)  10-28,  the 
development  of  the  Master  Capability  Library  (MCL)  or  Joint  Capability  Areas  (JCAs), 
or  the  follow-on  process  of  determining  appropriate  solutions  to  fill  the  capability  gaps 
(DAF,  2005:5).  A  further  discussion  of  the  CRRA  is  presented  in  Chapter  2. 

1.2  Implications 

Developing  architecture  models  to  evaluate  a  complex  system  can  lead  to  a  better 
understanding  of  the  system  and  its  behavior.  However,  not  all  models  are  equally  useful 
or  even  appropriate.  This  research  aims  to  equip  the  CRRA  practitioner  with  techniques 
to  improve  the  CRRA  methodology.  Identifying  ways  the  current  modeling  techniques 
can  be  extended  while  suggesting  other  architectural  models  not  currently  used  will  lead 
to  a  more  complete  picture  of  Air  Force  capability  performance.  It  will  result  in  a  more 
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accurate  analysis  of  current  capabilities  while  at  the  same  time  making  the  method  more 
defendable  to  those  questioning  its  results.  As  the  method  improves,  the  Air  Force  will 
increase  its  ability  to  ensure  it  is  equipped,  organized,  and  trained  to  deliver  the  necessary 
capabilities  to  the  Joint  force. 

1.3  Thesis  Overview 

With  the  research  questions  framed,  the  next  chapter  will  provide  a  more  detailed 
background  of  the  CRRA,  architecture,  and  the  PSMs  currently  used  to  support  the 
CRRA.  Chapter  3  will  explain  the  methodology  that  was  used  to  answer  the  research 
questions.  Chapter  4  will  present  and  explain  the  results  of  the  study.  Finally,  the 
conclusions  and  recommendations  will  be  presented  along  with  areas  for  further  research 
in  Chapter  5. 
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2.  Background 


In  the  2001  Quadrennial  Defense  Review,  a  new  strategy  was  presented  to 
determine  requirements  for  future  military  systems  or  organizational  changes  using  a 
Capabilities-Based  Assessment  (CBA)  approach  (DoD,  2001:17).  Shortly  thereafter. 
Secretary  of  Defense  Donald  Rumsfeld,  who  himself  had  grown  dissatisfied  with  the 
DoD’s  Requirements  Generation  System,  sent  the  below  memo  to  General  Peter  Pace, 
who  was  dual-hatted  as  Vice  Chairman  of  the  Joint  Chiefs  of  Staff  and  Chairman  of  the 


Joint  Requirements  Oversight  Council  (JROC)  (JCS,  2006:5). 


TO: 


CC: 


FROM: 


Gen.  Pace 

Paul  Wolf d  vrit  z 
Gen.  Myers 
Steve  Cambone 

Donald  Rumsfeld 


SUBJECT:  Requirements  System 


March  IS,  2002  7:17  AM 


As  Chairman  of  the  JROC,  please  think  through  what  we  all  need  to  do,  individually  or 
collectively,  to  get  the  requirements  system  fixed. 

It  is  pretty  clear  it  is  broken,  and  it  is  so  powerful  and  inexorable  that  it  invariably 
continues  to  require  tilings  that  ought  not  to  be  required,  and  does  not  require  things  that 
need  to  be  required* 

Please  screw  your  head  into  that,  and  leCs  have  four  or  five  of  us  meet  and  talk  about  it. 
Thanks, 


This  memo  led  to  an  extensive  redesign  of  the  methods  the  DoD  uses  to  determine 
future  capabilities.  In  fact,  it  was  decided  the  word  ‘requirements’  would  not  even  be 
used,  in  favor  of  the  word  ‘capability’.  It  was  out  of  this  work  that  the  Joint  Capabilities 
Integration  and  Development  System  (JCIDS)  was  born  and  began  implementation  by  the 
following  summer  of  2003  (JCS,  2006:5).  The  JCIDS  process  was  created  to  support  the 
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JROC  requirement  to  validate  and  prioritize  joint  warfighting  requirements  (CJCS, 
2007:2).  By  implementing  a  Capability-Based  approach  the  JCIDS  process  has 
integrated  joint  concepts  and  integrated  architecture  into  the  Doctrine,  Organization, 
Training,  Material,  Leadership  and  Education,  Personnel,  and  Facilities  (DOTMLPF) 
analysis  methodology  (CJCS,  2007:A-1).  It  attempts  to  achieve  this  by  leveraging  the 
experience  of  many  agencies  to  identify  improvements  to  existing  capabilities  as  well  as 
introduce  new  capabilities. 

Another  reason  for  the  development  of  a  Capabilities-Based  approach  was  the 
acknowledgement  that  the  United  States  could  no  longer  know  what  nation,  group  of 
nations  or  non-state  actor  would  pose  a  threat  to  the  United  States,  its  interests  or  allies. 
From  this  point  forward,  the  United  States  was  forced  to  try  to  anticipate  adversarial 
capabilities  and  prepare  an  adequate  response  to  those  threats.  In  contrast  to  a 
Requirements  Generation  System,  a  Capabilities-Based  model  focuses  on  how  a  range  of 
adversaries  might  fight  while  looking  at  various  contingency  locations  as  opposed  to  one 
static  threat.  It  relies  on  determining  what  capabilities  the  armed  forces  need,  to  either 
deter  or  defeat  its  adversaries  (DoD,  2001:IV). 

The  following  simplified  illustration  displays  the  change  in  concept  from  the  old 
Requirements  Generation  System  to  the  revised  Capability-Based  approach  (Figure  2.1). 
On  the  left  side  of  the  picture  is  the  old  “stove-piped”  version,  where  services 
individually  generated  their  system  requirements  from  scenarios  they  envisioned  as 
service-focused  missions,  rarely  assuming  any  capability  integration  with  the  other 
services.  It  wasn’t  until  the  end  of  the  process  that  an  attempt  was  made  to  integrate 
multi-force  capabilities,  taking  a  massive  amount  of  effort  from  the  joint  community. 
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Figure  2.1  Capability-Based  approach  (Jacques,  2006:4) 

This  differs  dramatically  from  the  right  side  of  the  illustration  where  capabilities 
are  conceptualized  in  a  “born  joint”  top-down  approach.  Here,  needed  capabilities  are 
identified  from  scenarios  derived  from  missions  based  on  joint  operational  concepts. 

From  concept  down  through  the  analysis,  the  assumption  is  that  all  operations  will  be 
joint  operations.  In  his  paper,  Bob  Larsen  commented  that  “The  resulting  warfighting 
capabilities  have  jointness  built  in;  they  start  that  way  and  they  finish  that  way”  (Larsen, 
2004:7).  During  the  analysis,  capability  needs  are  derived  to  determine  what  obligations 
must  be  fulfilled  to  complete  the  mission.  This  does  not  mean  that  the  individual  services 
no  longer  have  to  determine  their  own  requirements;  in  contrast,  “service  participation  is 
critical  to  developing  joint  perspectives  throughout  the  process”  (Larsen,  2004:7). 

Capability  Based  Assessments  are  not  unique  to  the  military;  many  commercial 
businesses  also  take  advantage  of  this  concept.  One  instance  of  this,  often  pointed  out  as 
a  success,  is  Wal-Mart  Corporation.  In  a  paper  by  Col  Stephen  Walker  on  Capabilities- 
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Based  planning,  the  differences  in  methodology  between  Wal-Mart  and  K-Mart  are 
discussed,  ultimately  leading  to  the  conclusion  that  a  Capabilities-Based  approach  is  a 
factor  (at  least  in  part)  to  Wal-Mart’s  rise  from  an  unknown  entity  in  the  late  1970s  to  a 
world  marketing  leader  at  the  present  time  (Walker,  2005:5). 

2.1  Implementing  JCIDS  with  the  CRRA 

To  implement  JCIDS,  the  Air  Force  created  their  own  Capabilities-Based 
Planning  (CBP)  process.  The  Air  Force  CBA  is  based  on  an  existing  Joint  Operating 
Concept  (JOC),  Joint  Integrating  Concept  (JIC)  or  Concept  of  Operations  (CONOPS).  Its 
purpose  is  to  identify  the  capabilities  required  to  execute  missions,  the  shortfalls  in 
existing  weapon  systems,  and  possible  “trade  space”  where  resources  can  be  taken  from 
areas  where  capabilities  are  comfortably  being  met  and  applied  to  areas  with  capability 
deficiencies.  The  CBA  is  scoped  by  six  elements:  Capabilities  desired,  Scenarios 
considered,  Functions  considered,  Types  of  solutions  considered,  Resource  limits,  and 
Planning  horizon  (JCS,  2006:22).  A  sub-process  to  CBP  is  the  Capability  Review  and 
Risk  Assessment  (CRRA).  The  CRRA  was  initiated  by  the  Chief  of  Staff  of  the  United 
States  Air  Force  (CSAF)  to  facilitate  the  development  of  an  operationally  focused 
Capabilities  analysis.  During  the  2002  Commander’s  Conference,  the  CSAF  outlined  his 
sight  picture: 

The  bottom-line  goal  for  the  CRRA  is  to  give  senior  USAF  leadership  an 
operational,  capabilities-based  focus  for  acquisition  program  decision¬ 
making.... To  accomplish  this  requires  reviewing  acquisition  programs  and 
discussing  disconnects  and  prioritization  in  relation  to  how  the  programs 
support  CONOPS  capabilities.  The  focus  shifts  from  program  review  to  a 
review  of  how  our  programs  contribute  to  the  warfighting  capability.  The 
CRRA  will  seek  to  evaluate  the  health  and  risk  of  required  CONOPS 
capabilities  over  the  next  20  years.  (HQ  USAF,  2007:2) 
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Originally  developed  in  2003,  and  published  biennially  thereafter,  the  CRRA’s 
purpose  is  to  analyze  the  capabilities  within  each  of  the  seven  Air  Force  CONOPS  which 
are: 


•  Global  Strike 

•  Homeland  Security 

•  Global  Mobility 

•  Global  Persistent  Attack 

•  Nuclear  Response 

•  Space  &  C4ISR 

•  Agile  Combat  Support 

The  CRRA  has  become  more  robust  with  each  iteration.  It  is  used  to  analyze  capabilities 
against  specifically  developed  joint  scenarios,  thus  assuring  current  and  future 
capabilities  requirements  and  shortfalls  are  addressed  with  respect  to  scenarios  of  interest 
to  the  joint  force.  The  CRRA  is  accomplished  with  respect  to  three  timeframes  to 
evaluate  current  capability  performance  as  well  as  expected  future  capability  performance 
(HQ  USAF,  2008a:  24).  The  CONOPS  Champions  at  Headquarters  Air  Force’s  office 
for  Capability  Based  Planning,  Operational  Planning,  Policy  and  Strategy  (AF/A5X-C), 
ensure  the  results  from  these  analyses  are  represented  in  the  Air  Force  Planning, 
Programming,  Budgeting,  and  Execution  (PPBE),  Strategic  Planning,  and  Joint 
Capabilities  Integration  and  Development  System  (JCIDS)  process  from  the  operational 
risk  perspective  (FAT,  2008). 

In  preparation  for  the  CRRA,  many  Air  Force  and  Joint  strategic  guidance 
documents  are  reviewed,  including  the  Joint  concepts,  Combatant  Commander 
(COCOM)  Integrated  Priority  Lists  (IPL),  the  Air  Force  Master  Capabilities  Library 
(MCL),  Joint  Capability  Areas  (JCA),  lessons  learned  from  recent  military  operations 
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(both  wartime  and  contingency)  and  the  shortfalls  from  the  previous  CRRA.  From  this 
detailed  review,  broad  lists  of  mission  areas  are  identified;  AF/A5X-C  focuses  on  the 
mission  areas  where  success  is  deemed  most  crucial.  During  the  CRRA  analysis,  two 
essential  questions  are  addressed:  “1)  what  is  the  Air  Force’s  overall  probability  of 
success  in  accomplishing  each  task,  and  2)  what  is  the  consequence  to  the  warfighting 
COCOM  due  to  a  particular  mission’s  probability  of  success?”  Together  the  answers  to 
these  questions  determine  the  risk  related  to  the  capability.  (FAT,  2008) 

2.2  CRRA  Analysis  using  PSMs 

The  CRRA  is  intended  to  be  an  analytically  supported  assessment  approach  that 
provides  the  flexibility  and  adaptability  required  to  meet  the  ever  changing  requirements 
of  the  Air  Force.  It  was  developed  to  be  compatible  with  the  Air  Force  Modeling  and 
Simulation  efforts,  and  to  be  relevant  across  the  entire  range  of  joint  operations  (HQ 
USAF,  2007:6). 

Beginning  with  the  2007  CRRA,  a  method  called  Process  Sequence  Modeling 
(PSM)  was  developed  and  employed  to  help  determine  the  most  efficient  and  effective 
use  of  limited  Air  Force  resources.  It  did  this  by  identifying  where  the  greatest  shortfalls 
and  highest  consequences  were  located  within  the  outlined  scenarios  (Bonafede,  2006). 

PSMs  are  the  primary  analytical  tool  for  the  CRRA  analysis  and  the  continuing 
Capabilities-Based  Assessment  Methodology  (CBAM).  PSMs  are  architecture-based 
activity  models,  consisting  of  a  series  of  activities  or  tasks,  each  represented  in  the  model 
as  a  “node”  (Bonafede,  2006:1).  The  degree  that  PSMs  are  based  on  larger  enterprise 
architectures  varies  between  the  CONOPS.  Within  the  Agile  Combat  Support  CONOPS, 
an  enterprise  architecture  based  on  DoDAF  version  1 .5  has  been  created.  This 
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architecture  currently  contains  an  All  Views  (AV)-l,  AV-2,  Operational  View  (OV)-l, 
OV-2,  OV-5,  and  OV-6c  with  future  releases  of  other  various  views  planed  (ACS 
Architecture  team,  2008: 1 1-12).  The  two  views  that  have  supported  the  development  of 
PSMs  most  directly  are  the  OV-5  Node  Tree  and  the  OV-6c  Operational  Event/Trace 
Description.  It  is  from  the  operational  activities  listed  in  the  OV-5  and  their  arrangement 
in  sequences  in  the  OV-6c  that  PSMs  are  developed  (Janus,  2008).  In  addition  to  using 
the  OV-5  and  sequencing  information,  it  has  been  shown  the  production  of  an  OV-2 
Operational  Node  Connectivity  Description  can  provide  a  means  to  graphically  show  the 
information  exchange  requirements  within  the  CONOPS.  This  can  provide  information 
about  the  interconnectivity  and  interdependence  of  the  capabilities  and  operations  under 
examination  (Eller,  2008:  26).  Eller  et  al.  also  developed  an  OV-5  to  PSM  traceability 
matrix  to  verify  PSM  construction  and  completeness.  Developing  enterprise  architecture 
as  the  basis  for  PSMs  ensures  they  are  traceable  to  strategic  guidance,  allows  for  reuse 
from  one  CRRA  to  the  next,  and  ensures  a  more  thorough  examination  of  the  capability. 

The  ability  to  accomplish  the  activities  in  a  PSM  for  the  purpose  of  achieving  a 
specific  effect  demonstrates  an  Air  Force  capability.  Therefore,  each  PSM  models  an  Air 
Force  capability.  Each  activity  is  a  step,  and  each  step  is  required  to  successfully 
complete  in  order  to  demonstrate  a  specified  capability.  Additionally,  the  ability  to 
accomplish  each  of  the  tasks  and  activities  in  the  PSM  represents  sub-capabilities  that 
enable  the  overall  PSM  capability.  In  the  2007  CRRA,  these  nodes  were  linked  to 
capabilities  within  the  Master  Capability  Library.  This  changed  for  the  2009  CRRA 
PSMs,  where  most  PSMs  link  nodes  to  the  Joint  Capability  Areas  (JCAs).  An  example 
of  a  PSM  is  presented  in  Figure  2.2.  The  goal  of  PSMs  is  to  bring  responsive,  repeatable 
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and  defendable  analytical  results  to  a  previously  haphazard  and  inefficient  capabilities 


analysis  (HQ  USAF,  2006). 


PSMs  are  similar  to  Functional  Flow  Block  Diagrams  (FFBDs),  long  used  in 
systems  engineering,  although  there  are  symbolic,  semantic,  and  syntax  differences.  A 
thorough  explanation  of  functional  analysis  and  FFBDs  is  provided  in  Appendix  A  of 
Systems  Engineering  and  Analysis  by  Blanchard  and  Fabrycky  (Blanchard,  2006:716). 
Within  the  Unified  Modeling  Language  (UML)  PSMs  are  similar  to  Activity  Diagrams. 
Examples  of  Activity  Diagrams  can  be  found  in  Applying  UML  and  Patterns  by  Craig 
Larman  (Larman,  2002:607).  Each  node  in  a  PSM  is  assigned  Probability  of  Success 
(P(S))  scores  by  Subject  Matter  Experts  (SMEs)  from  Air  Staff,  MAJCOMs,  Combatant 
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Commander  (COCOM),  direct  reporting  agencies,  and  other  services  in  conjunction  with 
AF/A5X-C.  SMEs  must  have  operational  experience  as  well  as  an  in-depth 
understanding  of  training  and  procedures  (HQ  USAF,  2008a:22).  One  CRRA 
practitioner  described  the  knowledge  and  experience  required  by  the  SMEs  by  stating 
“we  don’t  need  Subject  Matter  PARTICPANTS,  we  need  Subject  Matter  EXPERTS,  the 
“E”  is  what  is  important”  (FAT,  2008).  Whenever  possible,  SME  P(S)  are  backed  up  by 
supporting  documentation,  modeling  &  simulation,  lessons  learned  and  previous 
experience  (FAT,  2008).  After  a  PSM  node  is  scored,  the  overall  probability  of  success 
for  the  PSM  is  found  along  with  the  impact  each  node  has  on  the  overall  PSM  (HQ 
USAF,  2008a:26-32).  A  more  thorough  explanation  of  how  the  probability  of  success  is 
determined  is  presented  in  the  following  section.  While  the  overall  probability  of 
completion  is  important  to  understand,  the  more  meaningful  result  of  the  PSMs  is  to 
determine  the  specific  areas  or  tasks  that  are  potential  risk  drivers  for  the  completion  of 
the  selected  mission.  To  uncover  these  tasks  sensitivity  analyses  are  accomplished  on  the 
PSM  (HQ  USAF,  2007:17).  SMEs  also  determine  which  of  the  DOTMLPF 
fundamentals  influence  the  successful  completion  of  the  node  to  help  determine  the  root 
cause  of  the  problem  as  well  as  assist  with  the  later  step  of  determining  a  solution  to  close 
the  capability  gap,  if  one  exists  (HQ  USAF,  2008a:  22-23). 

Another  important  concept  developed  for  the  CRRA  is  the  determination  of 
Potential  Areas  to  Accept  Risk  (PAAR).  PAARs  are  mission  areas  that  are  performing 
better  than  required  and  therefore  offer  an  opportunity  to  shift  resources  from  PAARs  to 
areas  with  deficiencies.  Upon  discovery  of  a  PAAR,  primary  assessments  are  done  to 
determine  subsequent  order  effects  from  accepting  additional  risk.  PAARs  are  not 
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considered  solutions,  but  suggested  areas  of  additional  research  where  “trade  space” 
might  be  available  if  needed.  Where  PAARs  are  identified,  it  is  recommended  that 
further  analysis  be  done  prior  to  accepting  any  risk  (FAT,  2008). 

PSMs  were  first  developed  to  support  the  CRRA  (Bonafede,  2006).  However, 
they  have  been  successfully  used  to  assess  risk  for  other  purposes.  For  example,  while 
the  CRRA’s  purpose  is  to  evaluate  current  and  future  capability  performance  PSMs  can 
also  be  used  to  demonstrate  the  risk  associated  with  various  future  force  constructs.  Used 
in  this  way,  they  can  assist  with  selecting  the  appropriate  types  and  quantities  of  systems 
to  fill  the  capability  gaps  identified  by  the  CRRA  (Eller,  2008). 

2.3  Process  Sequence  Models 

This  research  studied  the  methods  used  to  develop  and  analyze  PSMs.  The 
assumptions  made  through  the  construction  and  evaluation  of  PSMs  are  addressed,  and 
the  applicability  of  those  assumptions  is  questioned.  Based  on  the  mission  they  are  trying 
to  model,  conclusions  are  made  as  to  the  strengths  and  weaknesses  of  the  current 
approach. 

A  PSM  is  a  basic  flow  block  diagram  of  the  tasks  or  steps,  which  need  to  be 
completed  to  successfully  accomplish  the  overall  mission.  An  excerpt  of  a  simple  PSM 
is  presented  in  Figure  2.3  (HQ  USAF,  2008a:  18). 
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Figure  2.3  PSM  Excerpt 

The  mission  tasks  are  represented  as  rectangles  while  the  arrows  between  the 
rectangles  show  the  sequenced  flow  of  the  process  (Bonafede,  2006:3).  The  arrow  also 
communicates  that  the  task  at  the  head  of  the  arrow  cannot  start  until  the  task  at  the  tail  of 
the  arrow  is  complete.  Diamond  nodes  included  in  the  PSMs  are  decision  nodes.  These 
nodes  direct  the  flow  through  the  model  based  on  whether  the  condition  in  the  decision 
node  is  satisfied.  Another  important  feature  of  a  PSM  is  a  parallel  process  that  allows 
steps  within  the  parallel  process  to  occur  at  the  same  time.  The  start  of  any  one  node  does 
not  require  the  completion  of  any  of  the  other  nodes  in  the  parallel  process.  However,  the 
process  cannot  move  beyond  the  parallel  process  until  all  nodes  are  complete.  An 
example  of  a  PSM  parallel  process  is  presented  in  Figure  2.4  (HQ  USAF,  2008a:  17-20). 
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Figure  2.4  PSM  Parallel  Process 


To  analyze  these  models  each  node  is  “scored”  by  a  team  of  SMEs.  The  score  is 
the  probability  the  node  will  be  successful  to  the  level  that  it  will  allow  success  for  the 
entire  PSM  (HQ  USAF,  2006:  13).  The  SMEs  are  presented  a  scenario  and  a  description 
of  the  node  they  are  to  score.  Included  in  the  description  are  assumptions  that  can  be 
made  and  factors  to  consider  while  determining  the  probability  of  success.  An  example 
of  the  description  the  SMEs  are  provided  to  assist  them  with  scoring  is  presented  in 
Figure  2.5.  This  description  is  from  a  node  in  the  Agile  Combat  Support  PSM  that 
examines  the  Air  Force’s  ability  to  identify  the  surge  in  personnel  (AF/A4LX,  2008:1 1). 


3.01.  Identify  Surge  Requirements:  Scored.  The  probability  of  success  of  accurately 
assessing  and  quantifying  need  for  increased  manpower  during  an  increase  in  operation 
tempo  (rate  of  effort)  to  meet  required  mission  dates  well  enough  and  within  the 
timeframe  needed  to  be  able  to  successfully  complete  this  PSM,  in  as  much  as  it  depends 
on  this  node.  Includes  capability  to  differentiate  and  prioritize  between 
AD/ARC/civilian/contractor  personnel.  (MCL  5 .6.4. 1 ) 


Figure  2.5  Example  PSM  Description 

Studying  the  description  shown  in  Figure  2.5  leads  to  some  questions  about  how 
the  SMEs  determine  the  score  they  provide.  Mainly,  the  score  is  described  as  “the 
probability  of  success  of  accurately  assessing  and  quantifying. . .”.  This  leads  to  the 
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question  ‘what  accuracy  is  required?’  Currently,  the  answer  to  that  question  is,  ‘The 
accuracy  that  allows  for  the  successful  completion  of  the  PSM’  and  is  left  up  to  the  SMEs 
to  decide.  Therefore,  SMEs  are  answering  two  questions.  The  first  is  ‘what  accuracy  is 
required  for  success?’  while  the  second  is  ‘what  is  the  probability  of  reaching  that  level?’ 
If  two  different  SMEs  determine  success  differently,  the  probability  of  reaching  that 
success  is  likely  to  be  different  as  well.  Additionally,  this  leads  to  the  question  of  how 
succeeding  at  different  levels  of  success  effects  the  outcome  of  a  PSM.  These  issues  are 
addressed  in  Chapter  4. 

For  the  purpose  of  focusing  on  only  one  node  at  a  time,  SMEs  are  told  to  score  the 
node  as  if  all  previous  nodes  were  successful  (HQ  USAF,  2008a:40).  Because  of  this,  the 
probability  of  success  SMEs  assign  to  the  node  is  actually  a  conditional  probability.  The 
score  is  the  probability  of  success  given  the  previous  nodes  were  all  successful. 

The  SMEs  actually  input  three  probabilities  to  each  node.  The  minimum 
expected  score  (worst  day),  the  most  likely  score,  and  the  maximum  expected  score  (best 
day).  This  creates  a  triangular  probability  density  function  (pdf)  where  the  independent 
variable  is  probability  of  success  as  shown  in  Figure  2.6  (Bonafede,  2006:7). 


P(s) 


Figure  2.6  Triangular  pdf  of  probabilities 
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Each  of  the  decision  nodes  is  provided  a  percentage.  The  percentage  represents 
the  expected  portion  of  occurrences  that  will  meet  the  condition  in  the  decision  node. 

The  decision  node  does  not  adjust  or  get  affected  by  any  events  that  precede  it.  It  has  no 
knowledge  of  events  that  occurred  before  it  and  simply  serves  to  provide  a  weighting  to 
the  nodes  that  come  after  it  based  on  the  percentage  of  occurrence  of  each  path.  In  this 
way  they  are  similar  to  an  uncertain  event,  sometimes  called  a  chance  node,  in  decision 
trees  (Holloway,  1979:32).  Once  the  nodes  are  populated,  a  Monte  Carlo  analysis  is 
performed  on  the  PSM.  A  Monte  Carlo  analysis  selects  a  probability  for  each  of  the 
nodes  based  on  the  triangular  distribution  entered.  It  then  analyzes  the  model  to 
determine  an  overall  probability  of  success  for  the  entire  PSM  (Bonafede,  2006:  4).  The 
simulation  is  then  run  again,  this  time  with  different  selected  values  for  each  of  the  nodes. 
This  simulation  is  run  approximately  2,000  times  to  find  the  most  likely  overall  PSM 
probability  of  success  values  (FAT,  2008). 

The  computations  required  to  find  the  overall  PSM  probability  of  success  depends 
on  the  configuration  of  the  model.  In  the  PSMs  currently  used  for  the  CRRA,  every  node 
needs  to  complete  successfully  for  the  PSM  to  complete  successfully.  Therefore,  the 
method  used  to  calculate  the  total  probability  of  success  is  simply  the  product  of  all  of  the 
probabilities  that  were  assigned  to  each  node.  Additionally,  even  if  tasks  are 
accomplished  in  parallel  with  each  other,  the  process  still  requires  each  to  be 
accomplished;  therefore  the  calculation  for  the  total  probability  remains  the  same. 

As  an  example,  the  overall  probability  of  success  for  the  two  processes  in  Figure  2.7  is 
identical  and  is  calculated  by  (HQ  USAF,  2007:  22): 

P(S) 

Total  —  Pa  (S)PB(S)PC(S)  (1) 
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where  P(S)a,  P(S)b,  P(S)c  are  the  probabilities  assigned  to  each  node  (Ebeling,  1997:  84). 


Figure  2.7  PSM  Computational  Equivalents 
Although  it  isn’t  seen  on  any  current  PSM  reviewed  for  this  research,  there  is 
nothing  to  preclude  a  model  from  having  redundant  tasks  (Bonafede,  2009).  A  redundant 
task  doesn’t  necessarily  need  to  be  accomplished  for  the  successful  completion  of  the 
process.  Since  every  task  does  not  need  to  be  completed,  the  algebra  describing  the  total 
probability  of  success  changes  to  equation  2.  An  example  of  redundant  task  PSM  is 
presented  in  Figure  2.8. 

P(S)Total  =  PA(S)[PB(S)  +  Pc(S)  -  PB(S)PC(S)  (2) 


Figure  2.8  PSM  Redundant  Nodes 


19 


For  situations  where  more  than  two  nodes  are  redundant  the  total  probability  can 
be  represented  with  the  expression  (Ebeling,  1997:  86): 

P(.S)rotal  =  (1  -  n?.i(l  -  PCS),))  (3) 

To  calculate  PSMs  with  decision  nodes,  the  equation  has  to  account  for  the 
change  in  analyzed  nodes  depending  on  the  path  the  process  follows.  For  example,  given 
the  process  model  in  Figure  2.9.,  if  60%  of  the  occurrences  result  in  a  YES  decision 
while  40%  of  the  occurrences  result  in  a  NO  decision,  then  the  closed  form  equation  for 
the  PSM  would  be  (FAT,  2008): 

Total  =  Pa(S)[C  6 )PB(S)PC(S)  +  (,4)Pd(5)P£(5)  (4) 


Figure  2.9  PSM  Decision  Node 


The  equations  for  the  overall  probability  of  success  for  the  PSM  presented  here 
assume  the  individual  steps  or  activities  are  independent  from  each  other  (Ebeling,  1997: 
84).  This  is  an  important  assumption  of  PSMs  that  will  be  addressed  in  Chapter  4. 

As  mentioned  above,  in  addition  to  the  calculation  of  the  total  probability  of 
success,  a  sensitivity  analysis  is  conducted  to  determine  the  node  that  has  the  highest 
impact  on  the  overall  probability  and  ultimately  the  consequence.  Two  different 
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sensitivity  analyses  are  conducted,  availability  and  incremental.  Originally  the 
incremental  technique  was  referred  to  as  the  marginal  sensitivity  measure  (Bonafede, 
2006:  6).  For  the  availability  method  the  probability  of  success  for  the  node  being 
examined  is  changed  by  half  of  the  difference  between  the  most-likely  probability  and  a 
probability  of  1.0  while  all  other  nodes  remain  at  their  most  likely  value.  For  example,  if 
the  node  was  scored  with  a  0.8,  the  node  probability  would  be  changed  to  0.9  whereas  if 
the  node  was  scored  0.9  the  score  would  be  changed  to  0.95.  Once  the  score,  or 
probability  of  success,  is  changed  for  one  node  the  total  PSM  probability  is  found  and 
related  to  operational  consequence.  That  node  is  then  returned  to  its  original  value,  the 
next  node  in  the  process  is  changed  and  the  total  probability  is  again  calculated.  The 
node  that  has  the  largest  impact  on  the  probability  of  success,  and  thus  the  consequence 
score,  will  be  listed  as  the  main  driver.  A  list  of  nodes  could  then  be  generated  in  order 
of  decreasing  impact  on  the  PSM  probability  of  success.  Two  factors  determine  how 
much  impact  any  particular  node  will  have.  The  first  is  the  node’s  orientation  in  the 
PSM.  If  the  node  is  accomplished  every  time  the  PSM  is  evaluated  it  will  have  more 
impact  than  a  node  positioned  after  a  decision  node  that  is  accomplished  only  20%  of  the 
time.  The  second  factor  is  the  value  of  the  node’s  original  most  likely  score.  All  other 
factors  being  equal,  a  node  originally  scored  0.2  is  going  to  be  a  larger  driver  than  a  node 
scored  0.8.  This  is  because  during  the  sensitivity  analysis  the  node  that  is  0.2  will  triple 
its  value  to  0.6  whereas  the  node  scored  0.8  will  only  change  to  0.9.  This  method  was 
decided  upon  by  the  CRRA  practitioners  to  highlight  the  nodes  that  have  a  lower 
probability  of  success  (FAT,  2008). 
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To  accomplish  the  incremental  sensitivity  analysis  the  first  partial  derivative  of  the 
closed  form  PSM  equation  is  computed  with  respect  to  each  node.  The  node  that  results 
in  the  greatest  partial  derivative  will  be  the  highest  risk  driver.  Again,  a  list  can  be 
computed  to  order  the  tasks  (or  nodes)  in  decreasing  order  of  sensitivity.  It  should  be 
noted  that  this  method  also  places  greater  weight  on  the  nodes  that  have  the  lowest 
probability  of  success.  To  see  why  this  is,  examine  a  PSM  with  three  nodes—  A,  B,  C. 
The  total  probability  is  P(S)x0tai=  P(S)a  P(S)b  P(S)c.  The  partial  derivatives  with  respect 
to  each  node  are: 

=  (P(S)g)(P(S)c)  =  (P(S)g)(P(S)c)  =  (P(S)g)(P(S)g) 

If  Pa  =0.8,  Pb=0.2  and  Pc=0.7  then  the  total  value  of  the  first  derivatives  would  be: 

dPtotal  _  ..  .  total  _  .  , 

dP(Sj~A~ '  ’  dP(S)B  "  -  ’  3P(S)  "  ' 

Node  B  would  be  listed  as  the  driver  because  it  has  the  highest  partial  derivative.  It  had 
the  highest  partial  derivative  because  it  had  the  lowest  probability  of  success  (FAT, 

2008). 

2.4  PSM  Assumptions 

Constructing  models  of  a  process  and  calculating  the  probability  of  success  using  this 
method  makes  certain  assumptions  about  the  system  or  process  it  is  modeling.  These 
assumptions  are  appropriate  if  the  real-world  system  meets  certain  criteria.  These  criteria 
are: 

•  Each  activity  or  task  represented  by  the  nodes  has  only  two  relevant  states;  either 
success  or  failure.  Another  way  to  describe  this  is  the  task  is  a  boolean  event 
whose  possible  states  are  mutually  exclusive. 
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•  If  an  activity  does  have  degrees  of  success,  the  degree  by  which  it  succeeds  does 
not  impact  the  chance  of  success  for  the  entire  PSM  or  for  any  other  node,  thus 
the  degrees  of  success  are  not  relevant. 

•  The  success  of  each  activity  can  only  be  dependent  on  whether  the  activities  that 
occur  before  it  were  successful  and  are  independent  in  all  other  ways.  The  score 
the  SME’s  give  each  node  is  the  probability  of  success  given  the  previous  nodes 
were  all  successful.  The  fact  that  nodes  can  only  be  dependent  on  the  nodes  that 
occur  before  them  implies  they  cannot  be  dependent  on  nodes  that  occur  in 
parallel  with  them.  For  the  bottom  process  in  Figure  2.7  node  B  could  not  depend 
on  node  C  or  vice  versa.  Additionally,  the  score  for  one  activity  will  not  affect 
the  score  on  another  activity;  therefore  the  scores  are  independent  from  each 
other. 

•  The  node  represents  an  activity  or  task  that  is  a  discrete  event  that  has  a  beginning 
and  end.  Without  a  specific  end  the  process  could  not  continue  to  the  next  node. 

•  The  nodes,  which  are  discrete  events,  represent  a  task  or  activity  in  the 
appropriate  timeframe  relative  to  other  tasks  or  activities.  For  example,  a  three 
node  PSM  consisting  of  a  node  representing  the  recruiting  of  new  personnel,  a 
node  representing  the  training  of  the  personnel,  and  a  node  representing  the 
personnel  performing  aircraft  maintenance  activities  is  to  be  evaluated.  If  the 
question  to  answer  is  ‘what  is  the  probability  the  airplane  can  be  adequately 
maintained  during  a  30-day  period  of  high  tempo?’,  the  probability  of  success 
entered  for  the  recruitment  of  new  personnel  shouldn’t  be  the  probability  to 
recruit  during  this  period  of  high  tempo  because  those  recruits  are  not  the  ones 
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performing  the  maintenance.  The  recruits  performing  the  maintenance  are  those 
who  joined  the  workforce  some  time  in  the  past.  Therefore,  to  answer  the 
question  of  ability  to  maintain  airplanes  during  the  30-day  window,  the 
probability  that  recruitment  was  done  successfully  in  the  past  would  have  to  be 
evaluated. 

When  examining  the  situations  that  are  currently  modeled  with  PSMs  to 
determine  if  they  meet  the  above  criteria,  what  should  be  remembered  is  that  all  models 
are  wrong;  some  are  useful  (Kurkowski,  2007).  Any  model  needs  to  find  the  correct 
balance  between  being  complex  enough  to  represent  reality,  while  remaining  simple 
enough  to  construct  and  analyze.  A  simpler  model  is  more  practical  to  build,  but  might 
not  be  as  realistic.  In  other  words,  an  attempt  to  make  the  model  "correct"  could  make 
the  model  less  useful  to  practitioners.  However,  committing  the  error  in  the  other 
direction  can  be  even  more  dangerous.  Making  assumptions  about  the  modeled  system 
for  the  purpose  of  simplifying  the  model  can  result  in  a  model  that  has  no  bearing  on 
reality  if  those  assumptions  are  not  valid.  The  model  could  reveal  supposed  insights  that 
do  not  correlate  to  reality  and  force  decisions  to  be  made  that  only  later  prove  to  be  poor. 
Constructing  models  that  either  require  too  much  information  or  ones  that  are  too 
simplistic  will  result  in  wasted  resources,  frustration  among  users,  and  can  lead  to  a 
situation  where  the  Air  Force  is  not  able  to  provide  a  capability  when  called  upon.  With 
those  warnings  in  mind,  an  evaluation  of  the  current  use  of  PSMs  is  presented. 

While  reviewing  the  PSMs  used  for  the  2007  CRRA,  as  well  as  a  sample  of  PSMs 
for  the  2009  CRRA,  some  questions  concerning  the  assumptions  were  noted.  For 
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example,  assuming  each  node  can  be  modeled  as  having  only  two  states,  success  or 
failure,  significantly  constrains  the  information  that  can  be  learned  from  the  analysis  and 
might  not  be  appropriate.  While  this  assumption  seems  appropriate  for  some  nodes,  other 
nodes  seem  to  be  better  served  if  they  are  allowed  to  undertake  other  various  states. 

Nodes  that  appear  to  meet  these  assumptions  well  are  those  that  represent  activities  that 
are  clearly  discrete  actions  in  time  and  either  result,  or  do  not  result,  in  an  outcome.  An 
example  of  this  is  a  node  that  represents  the  task  Deploy  Weapon.  It  is  reasonable  to 
assume  the  only  cases  that  matter,  or  need  to  be  examined,  are  if  the  weapon  deploys  or 
doesn’t  deploy,  while  any  case  in  between  can  be  correctly  placed  in  the  category  of  not 
deploying  successfully.  However,  there  are  other  activities,  many  of  them  residing 
within  combat  support,  for  which  the  degree  a  task  is  accomplished  is  important. 
Examples  of  nodes  that  seem  to  fall  squarely  into  this  category  are  ones  dealing  with 
training  or  the  assignment  of  personnel  to  a  particular  task,  where  quantities  are 
important.  There  are  many  other  areas  this  applies  to  as  well.  This  also  touches  on  a 
desire  heard  from  CRRA  practitioners  to  not  only  model  the  Air  Force’s  ability  to  deliver 
a  particular  capability  but  to  capture  what  level  of  quality  or  sufficiency  that  capability 
can  be  delivered  (FAT,  2008).  A  model  that  allows  certain  activities  of  interest  to  take  on 
an  independent  variable  other  than  probability  of  success,  and  to  allow  that  variable  to 
have  multiple  levels,  would  be  beneficial  to  the  CRRA  process. 

The  hypothesis  that  the  degree  to  which  a  task  is  accomplished  matters  to  the 
outcome  of  the  PSM  implies  that  the  degree  somehow  affects  the  results  of  other  nodes, 
and  consequently  the  whole  PSM.  This  challenges  another  assumption  already  listed, 
that  each  node’s  score  is  independent  from  other  nodes’  scores.  This  also  relates  to  the 
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desire  heard  during  the  course  of  the  research,  especially  among  the  Agile  Combat 
Support  community,  to  show  how  the  accomplishment  of  their  tasks  directly  contributes 
to  the  outcome  of  Operational  CONOPs  PSMs  (FAT,  2008).  To  accomplish  this,  a 
model  which  allows  individual  tasks  or  activities  to  map  to  and  influence  the  likelihood 
of  accomplishing  other  individual  tasks,  some  of  which  reside  across  many  different 
PSMs,  or  CONOPs,  would  be  beneficial. 

The  assumption  that  each  node  is  a  discrete  event  that  must  end  prior  to  the  next 
one  beginning  requires  a  more  nuanced  examination  to  determine  its  validity,  or  even  if 
the  assumption  is  actually  being  made  in  the  current  technique.  This  resides  in  the 
apparent  confusion  over  whether  or  not  these  models  are  in  fact  models  of  processes,  as 
their  name  would  suggest,  or  whether  they  are  more  akin  to  reliability  block  diagrams.  If 
it  is  a  true  process  model  then  the  order  of  the  nodes  does  matter  and  ensuring  the  nodes 
represent  activities  that  have  a  completion  is  important  to  moving  to  the  next  activity  in 
the  process.  However,  if  these  are  reliability  block  diagrams,  it  is  valid  to  have  a  series  of 
activities  all  occurring  at  the  same  time  that  will  only  complete  at  the  end  of  the 
examination  period.  A  simple  reliability  block  diagram  is  presented  in  Figure  2.10  for  an 
example  of  the  reliability  a  tire  will  stay  on  a  car  (Blanchard  and  Fabrycky,  2006:379- 
380).  One  “activity”  or  possibly  more  appropriate  “success  condition”  is  that  the  lug  nuts 
will  stay  on,  another  is  that  the  tire  stays  on  the  rim.  These  both  need  to  continuously 
happen  at  the  same  time  for  the  success  of  the  tire  staying  on  the  car. 


Figure  2.10  Example  Reliability  Block  Diagram 
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These  two  success  conditions  are  not  discrete  activities.  They  are  conditions  that  must  be 
maintained  simultaneously  during  the  entire  period  of  examination. 

There  are  tasks  within  PSMs  that  would  suggest  the  model  is  meant  to  be  a 
reliability  block  diagram;  however,  if  that  were  the  case  there  would  be  no  need  to 
include  AND  paths  that  attempt  to  show  mandatory  tasks  that  occur  at  the  same  time 
while  others  do  not.  Additionally,  there  does  not  appear  to  be  a  consensus  among  CRRA 
practitioners  as  to  the  answer  of  this  question.  The  reason  there  is  not  a  consensus  is  this 
does  not  matter  if  you  maintain  the  assumptions  listed  above,  mainly  that  the  tasks  are 
independent  from  each  other.  The  mathematical  model  to  represent  total  probability  of 
success  for  the  PSM  is  the  same  either  way.  As  described  in  the  sections  above,  it  would 
be  the  multiplication  of  the  probability  of  success  for  each  of  the  nodes.  However,  when 
relaxing  the  assumption  of  independence  it  becomes  necessary  to  understand  which 
nodes  represent  activities  performed  in  a  process  and  which  nodes  represent  activities  that 
are  ongoing  aspects  of  the  environment  within  which  the  process  is  being  run.  The 
timing  of  activities  also  need  to  be  considered  when  dependence  is  added  to  the  model  to 
ensure  that  prior  to  the  execution  of  a  node,  all  nodes  it  is  dependent  upon  have  executed. 
This  leads  to  the  final  caution  that  the  nodes  are  both  modeled  and  evaluated  by  SMEs  in 
the  appropriate  time  frame  relative  to  each  other. 

2.5  Consequence  Values 

The  second  part  of  the  CRRA  process  is  determining  the  consequence  associated 
with  the  output  of  the  PSM.  The  PSM  probability  of  success  is  linked  to  operational 
consequences  through  a  methodology  and  excel  spreadsheet  tool  developed  at  AF/A5X- 
C.  To  accomplish  this,  members  of  AF/A5X-C  meet  with  the  Combatant  Commands  to 
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determine  how  the  performance  of  the  PSMs  affect  their  ability  to  complete  operational 
missions. 

The  first  step  is  to  elicit  from  the  COCOMs  what  are  the  Good  Enough  Value 
(GE)  and  the  Limited  Military  Value  (LMV)  PSM  probabilities  (Bonafede,  2006:  2).  GE 
is  defined  as  “The  probability  of  success  for  a  particular  mission,  above  which  the 
warfighter  has  enough/excess  capability  to  successfully  complete  mission  objectives 
within  acceptable  levels  of  consequence”.  The  LMV  is  defined  as  “The  probability  of 
success  for  a  particular  mission,  below  which  is  negligible  military  value  to  the 
warfighter”.  These  regions  are  placed  onto  a  graph  as  shown  in  Figure  2.1 1  (FAT,  2008). 
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Figure  2.1 1  Consequence  Value  Ranges 

Next,  COCOMs  select  severity  factors  to  consider  for  the  particular  mission  being 
executed.  Severity  factors  are  areas  of  concern  that  should  be  considered  to  understand 
the  full  range  of  consequences.  Examples  of  severity  factors  are  friendly  casualties, 
public  security  confidence,  loss  of  friendly  infrastructure,  and  adverse  economic  impact, 
to  name  a  few  (Bonafede,  2006:  5).  AF/A5X-C  has  developed  16  severity  factors,  from 
which  COCOMs  select  2-5  they  feel  are  most  relevant  to  the  mission  under  examination. 
Each  severity  factor  is  then  divided  into  6  consequence  values  pertaining  to  catastrophic, 
extensive,  major,  substantial,  modest,  and  minor  outcomes.  Descriptions  of  each 
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consequence  value  are  provided  in  a  word  picture  to  adequately  provide  examples  of 
what  would  constitute  that  value.  The  Loss  of  Friendly  Infrastructure  description  is 
shown  in  Table  2.1. 


Table  2.1  Example  of  Consequence  Description 


Severity  Factors  > 

Loss  of  Friendly  Infrastructure 

Catastrophic  (6) 

Friendly  centers  of  gravity  damaged  or  destroyed. 
Widespread  damage/loss  of  critical  infrastructure. 
Nationwide  electrical  power  outages  for  indeterminate 
amount  of  time;  nationwide  water,  food,  and  medicine 
supply  shortages  affect  most  citizens. 

Extensive  (5) 

Large  regional  electrical  power  outages  for  indeterminate 
amount  of  time;  nationwide  water,  food,  and  medicine 
supply  shortages. 

Major(4) 

Friendly  centers  of  gravity  attacked.  Local  damage/  loss 
of  critical  infrastructure.  Regional  infrastructure  affected. 
National  transportation  system  damage  (such  as 
widespread  airport  shut  downs)  for  indeterminate  amount 
of  time  ,  large  regional  extende 

Substantial  (3) 

Critical  infrastructure  sustains  local  damage.  No  regional 
damage  or  loss.  Nat'l  transportation  system  temporarily 
ceases,  regional  extended  electrical  power  outages, 
nationwide  Internet  operation  tern porarily  incap acitated , 
National/international  banking 

Modest  (2) 

Local/  limited  damage  to  infrastructure.  No  regional 
damage  or  loss.  National  transportation  system 
significantly  slowed  down,  widespread  Internet  operation 
significantly  slowed,  National/international  banking  and/or 
monetary  systems  s ignificantty  s lowed 

Minor  (1> 

No  loss  of  critical  infrastructure.  Nat'l  transportation 
system  has  temporary  slow  downs  or  slow  downs  only 
limited  to  regional  activity,  Internet  operation  has  regional 
slow  downs,  regional  banking  and/or  monetary  systems 
significantly  slowed,  regional  s 

Following  the  selection  of  the  severity  factors,  the  COCOMs  determine  what  the 
worst  case,  most  likely,  and  best  case  consequence  values  are  for  each  of  the  three 
probability  regions  identified  in  the  first  step  (below  LMV,  between  LMV  and  GE,  and 
above  GE)  as  shown  in  Table  2.2. 


29 


Table  2.2.  Consequence  Scoring 


From  the  information  gathered  and  a  relative  weight  to  each  of  the  severity  factors 
selected,  a  graph  is  generated  shown  in  Figure  2.12. 
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Figure  2.12  COCOM  Consequence  Curve 

This  graph  relates  the  overall  probability  of  success  for  a  PSM,  shown  on  the  horizontal 
axis,  to  consequence  values,  shown  on  the  vertical  axis.  For  each  run  of  the  PSM  a  P(S) 
is  generated  that  relates  to  a  consequence  value  through  the  chart  above.  As  stated 
earlier,  this  is  run  approximately  2,000  times  to  find  the  consequence  of  the  PSM. 
Additionally,  during  the  sensitivity  analysis  the  node  that  creates  the  greatest  change  in 
consequence  score  is  determined  to  be  the  driving  node  (HQ  USAF,  2008b)  (FAT,  2008). 
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This  analysis  completes  the  analytic  portion  of  the  CRRA  evaluation  process,  evaluating 
both  the  capability  gaps  as  well  as  the  consequences  to  the  warfighter  if  a  capability  gap 
goes  unfilled. 

The  score  calculated  by  the  analytic  method  described  above  is  only  part  of  the 
assessment.  There  is  also  supplementary  information  elicited  from  the  SMEs  in  the  form 
of  a  rationale  to  justify  the  P(S)  they  assigned.  During  interviews  with  several  SMEs  and 
stakeholders  it  was  purported  that  the  rationale  given  was  actually  more  important  than 
the  probability  scores.  The  rationale,  it  was  reasoned,  was  the  basis  of  the  capabilities’ 
performance  and  any  underlying  gaps  that  could  be  associated  with  a  probability  of 
failure.  However,  during  a  SME  scoring  conference,  there  seemed  to  be  significant 
confusion  concerning  what  constituted  node  failure  instead  of  a  somewhat  ambiguous 
idea  of  milder  capability  degradation.  This  question  was  pondered  by  many  in 
attendance,  but  was  not  addressed  in  the  conference,  with  hopes  that  future  iterations  of 
CRRA  analysis  could  extend  to  a  level  where  varied  levels  of  success  can  be  considered. 
2.6  Architecture 

While  architecture  in  its  classical  form  as  the  art  and  science  of  designing  and 
erecting  buildings  has  been  a  discipline  for  thousands  of  years,  the  idea  of  general 
systems  architecture  and  enterprise  architecture  are  much  more  recent  (Architecture, 
2000).  As  a  tool,  architecting  becomes  useful  when  trying  to  define,  describe,  and 
understand  complex  and  unprecedented  systems.  As  a  system  grows,  the  number  of 
interrelationships  grows  at  a  much  faster  pace  than  the  number  of  components;  this 
requires  tools  and  techniques  to  understand  the  various  aspects  of  the  system  (Maier  and 
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Rechtin,  2002:5).  In  a  book  on  Service  Oriented  Architecture,  the  authors  describe  the 

evolution  and  need  for  architecture  in  this  way: 

“In  the  beginning,  there  were  programs,  and  programs  were  good,  and  programs 
didn  ’t  need  no  stinking  architectures.  And  then  there  was  business  and  the 
businesses  grew,  and  the  programs  grew,  and  chaos  was  on  the  face  of  the  business. 

And  so,  in  an  effort  to  create  order,  programmers  adopted  systematic  structures  to 
organize  the  programs  and  help  the  business.  And  any  structure,  be  it  a  strip  mall  or 
the  Taj  Mahal,  or  even  Noah ’s  Ark,  has  some  underlying  design,  however 
haphazard,  know  as  an  architecture.  ”  (Hurwitz,  2006) 

Through  this  somewhat  humorous  explanation  of  why  architecture  is  important,  it 
becomes  obvious  that  as  projects  become  bigger,  architecture  can  help  maintain  order, 
focus,  and  direction.  This  same  phenomenon  occurred  during  the  creation  of  classical 
architecting.  Thousands  of  years  ago  the  Greeks  and  the  Romans  built  cities,  aqueducts, 
road  systems,  and  defenses.  The  increasing  complexity  of  the  systems  they  created  led  to 
the  formulation  of  new  tools  to  understand  them  (Rechtin,  1991  :xiii).  The  current 
practices  in  systems  architecting  have  their  root  in  the  mid-1980’s  (GAO,  2003:1). 

During  this  time  there  was  another  increase  in  the  complexity  of  systems  under  design, 
this  time  in  the  form  of  information  technology  (Rechtin,  1991  :xiv). 

2.7  Architecture  Frameworks 

With  the  maturation  of  system  architectures,  various  architectural  frameworks 
have  been  developed.  A  framework  is  a  standard  by  which  architectures  are  described 
and  is  analogous  to  blueprint  standards  (Maier  and  Rechtin,  2002:221).  Current 
frameworks  can  be  traced  to  the  late  1980s.  It  was  at  that  time  that  John  Zachman  wrote 
an  article  published  in  IBM  Systems  Journal  titled  “A  Framework  for  Information 
Systems  Architecture”.  This  paper  laid  the  foundation  for  what  became  known  as  the 
Zachman  framework  which,  according  to  the  federal  government’s  Chief  Information 
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Officer’s  Council,  “has  received  worldwide  acceptance  as  an  integrated  framework  for 
managing  change  in  enterprises  and  the  systems  that  support  them”  (CIO  Council, 
1999:19).  In  his  seminal  work,  Zachman  examined  classical  architecture  to  uncover  a 
descriptive  framework  that  could  be  used  to  describe  information  systems  (Zachman, 
1987).  He  described  two  architectural  “observations”  that  became  the  basis  for  his 
framework  and  subsequent  frameworks.  The  first  is  that  the  architecture  has  different 
representations.  These  were  called  “fundamental  architectural  representations”  while 
contemporary  frameworks  have  called  these  representations  views  or  viewpoints.  He 
explained  there  are  three  representations  based  on  the  perspective  of  the  owner,  designer, 
and  builder  of  the  system.  An  important  distinction  made  by  Zachman  was  that  the  three 
views  are  not  merely  descriptions  of  the  system  at  varying  levels  of  detail  but  are  actually 
different  in  nature  and  content  showing  the  same  system  from  a  different  point  of  view. 
Level  of  detail  is  an  independent  variable  and  can  exist  at  any  desired  level  within  each  of 
the  representations.  The  second  observation  was  that  within  each  representation  there  are 
different  descriptions  that  answer  the  six  interrogatives,  who,  what,  where,  why,  when, 
and  how.  Even  though  each  of  these  descriptions  addresses  the  same  system,  they  are 
independent  from  each  other  and  provide  different  information  about  the  system.  The 
combination  of  the  representations  (or  views)  and  descriptions  into  a  matrix  forms  the 
foundation  of  the  Zachman  framework  (Zachman,  1987:282).  A  more  recent  example  of 
the  Zachman  Framework  that  incorporates  more  than  three  views  is  presented  in  Table 
2.3. 
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Table  2.3.  Zachman  Framework  (Ambler,  2004) 


What 

(Data) 

How 

(Function) 

Where 

(Locations) 

Who 

(People) 

When 

(Time) 

Why 

(Motivation) 

Scope 

{contextual} 

Planner 

List  of  things 
important  to  the 
business 

List  of  processes 
that  the  business 
performs 

List  of  locations  in 
which  the 
business 
operatses 

List  of 

organizations 
important  to  the 
business 

List  of  events/ 
cycles  important 
to  the  business 

List  of  business 
goals/strategies 

Enterprise  Model 
{conceptual} 

Business  Owner 

e.g.  Semantic 
Model 

e.g.  Business 
Process  Model 

e.g.  Business 
Logistics  System 

e.g.  Workflow 
Model 

e.g.  Master 
Schedule 

e.g.  Business 
Plan 

System  Model 
{logical} 

Designer 

e.g.  Logical  Data 
Model 

e.g.  Application 
Architecture 

e.g.  Distributed 
System 
Architecture 

e.g.  Human 
Interface 
Architecture 

e.g.  Process 
Structure 

e.g.  Business 
Rule  Model 

Technology  Model 
{physical} 

Implementer 

e.g.  Physical  Data 
Model 

e.g.  System 
Design 

e.g.  Technology 
Architecture 

e.g.  Presentation 
Architecture 

e.g.  Control 
Structure 

e.g.  Rule  Design 

Detailed  Representation 
{out-of-context} 

Subcontractor 

e.g.  Data 
Definition 

e.g.  Program 

e.g.  Network 
Architecture 

e.g.  Security 
Architecture 

e.g.  Timing 
Definition 

e.g.  Rule 
Definition 

Functioning  System 

e.g.  Data 

e.g.  Function 

e.g.  Network 

e.g.  Organization 

e.g.  Schedule 

e.g.  Strategy 

Several  other  frameworks  have  been  developed  since  Zachman  proposed  his  in 
1987  to  include  frameworks  by  the  Institute  of  Electrical  and  Electronics  Engineers 
(IEEE),  the  International  Standards  Organization  (ISO)  and  the  DoD  (Maier  and  Rechtin, 
2002:222).  The  DoD  released  its  first  framework  in  1996  in  response  to  passage  of  the 
Information  Technology  Management  Reform  Act  of  1996,  later  designated  the  Clinger- 
Cohen  Act  (CCA)  of  1996  (DoD,  2007:3-2).  The  CCA  mandates  “An  integrated  DoD 
architecture  with  operational,  system,  and  technical  views  shall  be  developed,  maintained 
and  applied. . .”  (DoD  CIO,  2006: 168).  As  the  original  name  of  the  act  suggests,  this 
applied  to  information  technology  management.  In  June  1996  the  DoD  released  the 
Command,  Control,  Communication,  Computer,  Intelligence,  Surveillance  and 
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Reconnaissance  (C4ISR)  Architecture  Framework  version  1.0.  That  Framework  was 
updated  and  released  as  C4ISR  Architecture  Framework  version  2.0  in  December  of 
1997.  Two  months  later  the  DoD  mandated  the  framework  be  used  for  all  C4ISR 
architecture  descriptions  (DoD,  2007:ES-3).  The  framework  continued  to  expand  and  in 
2003  was  renamed  to  the  DoD  Architecture  Framework  (DoDAF)  to  capture  the  intent  to 
use  this  framework  for  not  just  IT  systems  but  for  all  DoD  systems  to  included  enterprise 
architecting. 

The  Open  Group  Architecture  Framework  (TOGAF)  defines  architecture 
framework  as: 

“a  tool  which  can  be  used  for  developing  a  broad  range  of  different  architectures.  It 
should  describe  a  method  for  designing  an  information  system  in  terms  of  a  set  of 
building  blocks,  and  for  showing  how  the  building  blocks  fit  together.  It  should 
contain  a  set  of  tools  and  provide  a  common  vocabulary.  It  should  also  include  a  list 
of  recommended  standards  and  compliant  products  that  can  be  used  to  implement 
the  building  blocks.  ”  (TOGAF,  2008) 

To  that  end,  the  research  focused  on  three  of  what  was  understood  to  be  the  most  relevant 

architectural  frameworks  for  this  project.  The  DoDAF  Version  1.5,  the  Ministry  of 

Defense  Architectural  Framework  (MODAF)  and  the  Draft  DoDAF  Version  2.0  were 

analyzed  in  detail.  As  previously  mentioned,  there  are  several  other  types  of  architecture 

frameworks  available,  however  this  research  was  focused  toward  these  particular 

frameworks  due  to  the  intended  audience. 

DoDAF,  which  has  been  the  standard  DoD  guidance  since  it  was  initially 
approved  in  2003  as  Version  1.0,  later  upgraded  in  2007  to  Version  1.5,  describes  29 
architectural  models  that  provide  guidance  to  consistently  represent  different  aspects  of  a 
system  depending  on  the  information  needed  to  be  displayed. 
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The  MODAF,  used  by  the  United  Kingdom,  adopted  the  DoDAF  v  1.5  views. 

The  All  View  (AV),  Operational  View  (OV),  Systems  and  Services  View  (SV),  and  the 
Technical  Standards  View  (TV),  were  upgraded  by  adding  Strategic  Views  (StVs), 
Service  Oriented  Views  (SOVs),  and  Acquisition  Views  (AcVs). 

In  comparison,  the  Draft  DoDAF  v  2.0  that  was  reviewed  for  this  research  has  a 
total  of  49  different  models  to  display  the  various  representations.  These  are  now  called 
“Viewpoints”  rather  than  “Views”.  It  incorporated,  among  other  things,  similar  views  to 
those  found  in  the  MODAF.  For  example,  the  Capability  Viewpoint  (CV)  can  be  directly 
compared  with  MODAFs  StV,  where  “The  viewpoints  within  the  CV  are  high-level  and 
describe  capabilities  using  terminology  which  is  easily  understood  by  decision  makers 
and  used  for  communicating  a  strategic  vision  regarding  capability  evolution”  (DoD, 
2008a:25).  Similarly,  the  updated  Services  Viewpoint  can  be  related  to  the  MODAF 
SOV,  which  supports  the  operational  and  capability  functions.  Furthermore,  the  Project 
Viewpoint  was  developed  using  feedback  from  the  Acquisition  community  and  can  be 
compared  to  the  MODAF  AcV.  For  ease  of  reference,  the  following  pictorial 
representations  have  been  provided  so  the  reader  may  compare  between  the  Draft 
DoDAF  v  2.0  and  the  MODAF. 
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(MODAF,  2008) 


Figure  2.13  DoDAF  and  MODAF  Views 
Additional  viewpoints  were  added  or  updated  to  enhance  the  capabilities  of  the 
DoDAF  architecture.  For  example,  in  the  DoDAF  v  1.5,  Services  and  Systems  Views 
were  linked  together,  whereas  in  the  Draft  DoDAF  v  2.0  they  are  separated  into  distinct 
Viewpoints.  Furthermore,  a  Data  and  Information  Viewpoint  was  created  using  OV-7 
and  SV-11  views  of  DoDAF  v  1.5.  Figure  2.17  provides  a  pictorial  overview  of  how  the 
updated  Draft  DoDAF  v  2.0  compares  with  the  older  DoDAF  v  1.5. 
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DoDAF  VI  .5  Evolution  to  DoDAF  V2.0 


(DoDAF,  2008b:  127) 

Figure  2.14  DoDAF  Evolution 

Since  the  creation  of  the  DoDAF  in  2003  there  has  been  effort  to  transition  from  a 
product-centric  framework  to  one  that  is  data-centric.  For  example,  as  opposed  to 
focusing  on  the  viewpoint  and  description  of  a  particular  architecture  product,  the 
emphasis  would  be  on  creating  a  common  database  of  architectural  information  from 
which  various  products  can  be  built.  By  changing  the  focus,  the  creation  of  architectures 
that  share  common  data  descriptions  across  several  viewpoints  becomes  easier  to 
construct,  a  result  referred  to  as  an  integrated  architecture.  Additionally,  models  can  be 
generated  that  don’t  necessarily  fit  into  one  of  the  viewpoints  described  within  DoDAF. 
2.8  Conclusion 

This  chapter  provided  background  to  the  Air  Force’s  Capability  Based 
Assessment,  specifically  the  CRRA,  introduced  Process  Sequence  Models,  and  explained 
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how  the  probability  of  success  is  related  to  mission  outcomes  and  consequences.  This 
chapter  also  provided  background  to  Enterprise  and  Systems  Architecture  and 
Architecture  Frameworks.  Additionally,  changes  to  the  DoDAF  in  the  draft  version  2.0 
that  were  deemed  important  to  this  research  were  presented.  An  understanding  of  the 
basics  of  these  topics  greatly  assists  in  understanding  the  analysis  and  suggestions 
presented  in  Chapter  4  and  the  conclusions  presented  in  Chapter  5.  Next,  the 
methodology  that  was  used  to  answer  the  research  questions  and  develop  research 
conclusions  is  presented  in  Chapter  3. 
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3.  Methodology 


3.1  Overview 

To  answer  the  first  research  question  listed  in  Chapter  1,  “How  is  architecture 
currently  being  used  to  support  the  CRRA?”,  Process  Sequence  Models  (PSMs)  as  they 
are  currently  performed  have  been  explained  along  with  the  assumptions  they  make  about 
the  system  they  model.  It  was  found  PSM  assumptions  force  limitations  on  the  model 
that  would  be  beneficial  to  remove.  Additionally,  those  assumptions  might  not  even  be 
appropriate  for  some  situations  currently  modeled  by  PSMs.  This  chapter  introduces  a 
methodology  to  relax  those  assumptions.  Following  this  chapter,  Chapter  4  provides 
more  details  on  the  new  methodology  and  demonstrates  its  application. 

When  answering  the  second  research  question,  “How  can  the  current  architectural 
models  be  extended  to  make  them  more  appropriate  and  useful?”,  it  was  an  aim  of  this 
research  to  keep  any  proposed  architectural  models  based  on  the  current  techniques  of 
PSMs.  This  is  to  prevent  the  CRRA  practitioners  from  having  to  start  from  scratch  if 
they  choose  to  adopt  the  conclusions  of  this  research.  Consequentially,  it  allows  the 
CRRA  practitioner  to  make  incremental  improvements  to  their  modeling  techniques  and 
therefore  increase  the  chance  the  suggestions  are  considered  and  implemented.  The  other 
reason  the  research  aimed  to  extend  PSMs  as  opposed  to  completely  deviating  from  them 
is  that  PSMs  do  have  value  that  has  been  proven  in  actual  use  for  years.  Any  critique  of 
the  current  technique  should  not  be  construed  to  mean  the  research  has  determined  PSMs 
have  not  been  useful  to  the  purpose  they  have  been  applied. 

The  improved  model  suggested  by  this  research,  called  Extended  Sequence 
Models  (ESMs),  will  continue  to  result  in  an  overall  probability  of  success  score  that  can 
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be  attributed  to  an  operational  consequence  level  identical  to  the  current  PSM  method. 
The  probability  of  success  will  be  determined  by  listing  all  activities  necessary  for  the 
demonstration  of  a  capability,  also  identical  to  the  PSM  technique.  ESMs  will  continue 
to  require  Subject  Matter  Expert’s  (SMEs)  knowledge  and  judgment  to  predict  the 
performance  of  activities  within  an  overall  scenario  and  those  performances  will  be 
captured  in  probability  density  functions  (pdf).  Identical  sensitivity  analysis,  availability 
and  incremental,  can  be  performed  on  the  ESM  as  performed  on  the  PSMs.  Additionally, 
the  SMEs  will  still  provide  rationale  for  the  scores  they  provide,  which  is  one  of  the  most 
important  steps  in  the  current  PSM  technique. 

To  the  maximum  extent  possible,  the  only  difference  between  PSMs  and  ESMs 
are  those  portions  specifically  improved.  The  first  improvement  allows  activities  to  take 
on  metrics  other  than  success  or  failure,  such  as  metrics  of  quality  or  sufficiency.  The 
second  improvement  allows  an  activity’s  pdf  to  adjust  based  on  the  outcome  of  other 
nodes  which  it  depends.  This  relationship  is  approximated  using  the  techniques  of  linear 
regression  to  find  the  least  squares  fit  for  the  data  provided  by  the  SMEs.  These 
dependencies  are  completely  determined  by  the  SMEs  and  the  CRRA  practitioners  to 
focus  on  areas  they  feel  are  most  important.  Third,  ESMs  improve  PSMs  by  explicitly 
acknowledging  that  not  all  nodes  represent  specific  activities  to  be  accomplished,  rather 
some  represent  environmental  factors  that  are  dictated  by  the  scenario  and  change  the 
ability  to  perform  certain  activities.  ESMs  are  explained  fully  in  the  next  chapter  by 
presenting  the  technique  in  a  step  by  step  process. 

To  answer  the  third  research  question,  “What  other  architectural  models  can  be 
built  and  analyzed  that  would  assist  the  CRRA?”,  an  understanding  of  the  current  method 
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was  acquired  through  reading  CRRA  documentation,  interviewing  CRRA  stakeholders, 
and  attending  a  PSM  scoring  session.  From  there,  three  architectural  models  were 
decided  upon  that  would  allow  the  SMEs  to  provide  more  accurate  scores  to  ESM  nodes 
while  at  the  same  time  making  the  scores  they  provide  more  defendable.  The  creation  of 
these  architectural  models  is  “Step  0”  of  the  ESM  construction  technique.  They  are 
developed  for  the  purpose  of  providing  the  SMEs  and  CRRA  practitioners  the  most 
detailed  and  accurate  information  possible  to  increase  the  reliability  and  defendability  of 
the  CRRA. 

3.2  Summary 

With  an  overview  of  the  methods  used  to  answer  the  research  questions  presented, 
the  next  chapter  explains  in  detail  the  outcome  of  the  method  and  provides  examples  of 
their  implementation.  Finally,  Chapter  5  presents  conclusions,  recommendations,  and 
areas  for  further  research. 
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4.  Analysis  and  Results 


4.1  Introduction 

This  chapter  begins  with  an  explanation  of  architectural  models  that  can  be 
developed  at  the  commencement  of  the  Capability  Review  and  Risk  Assessment  (CRRA) 
to  improve  Extended  Sequence  Model  (ESM)  construction,  scoring,  and  execution. 
Following  this  explanation,  the  ESM  technique  is  presented.  To  conclude  the  chapter,  the 
ESM  technique  is  tested  by  applying  it  to  a  portion  of  an  Agile  Combat  Support  (ACS) 
Process  Sequence  Model  (PSM)  currently  in  use  to  support  the  2009  CRRA. 

4.2  Architecture  Framework  Analysis 

While  there  are  exceptions,  most  DoD  processes  do  not  require  the  use  of  any 
specific  architectural  view  or  product.  Even  though  they  are  not  always  required,  the  use 
of  a  variety  of  views  and  products  can  streamline  product  deployment  and  reduce 
confusion.  There  are  various  examples,  such  as  DoD  Directive  5000.02  and  CJCSI 
3170.01F  Joint  Capabilities  Integration  and  Development  System  (JCIDS),  which  signify 
the  importance  of  providing  architecture  information  by  requiring  specific  views  for 
different  development  stages  of  a  system.  By  using  a  well  developed  and  thoroughly 
planned  architecture,  the  CRRA  analysis  can  be  more  understandable,  more  readily 
accepted,  and  more  defendable  under  scrutiny. 

A  significant  concern  of  many  individuals  and  teams  alike  is  that  building 
architecture  is  both  time  consuming  and  ineffective.  This  is  partially  due  to  the  mistaken 
belief  that  architecture  views  are  built  for  their  own  sake  and  that  they  are  an  end  product 
instead  of  an  analysis  tool  to  help  the  user.  Volume  I  of  the  Draft  DoDAF  v  2.0  stresses 
this  concept  by  stating 
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“Architecture  views  (formerly  ‘products  ’)  are  no  longer  the  end  goal,  but  are 
described  solely  to  facilitate  useful  access  to  information  in  the  architecture 
database.  All  views  are  tailorable.  The  requirements  for  data  completeness  and 
self-consistency  within  the  data  schema  are  more  critical  than  the  view  chosen  at 
any  particular  time  by  a  particular  user.  Analytics,  properly  conducted, 
represent  a  powerful  tool  for  the  decision-maker,  ensuring  that  the  most 
appropriate  and  current,  as  well  as  valid  data  is  used  for  decision-making” 

(DoD,  2008a: 82). 

It  also  takes  this  idea  further  by  asserting  “Architectures  well  designed,  and  consistent 
with  the  purpose  for  which  they  were  created,  are  well  suited  to  the  analytic  process” 
(DoD,  2008a:83).  A  complete  list  of  suggested  architectural  views  and  their 
representations  may  be  found  in  volume  2  of  the  Draft  DoDAF  v  2.0. 

As  part  of  this  research,  many  of  the  CRRA  stakeholders  were  interviewed 
concerning  their  knowledge  and  opinion  of  the  architecture  that  surrounds  the  CRRA, 
and  more  specifically,  the  PSM  analysis.  Some  had  the  mistaken  belief  that  architecture 
was  simply  limited  to  DoDAF  views,  while  others  believed  that  using  DoDAF  or  any 
other  formal  architecture  framework  would  be  a  hindrance  to  the  PSM  analysis  as  they 
considered  the  architecture  too  “inflexible”  to  be  useful.  One  stakeholder  even  went  so 
far  as  to  state  how  they  had  to  “divorce  themselves  from  the  architecture”  to  develop  a 
useful  analysis  tool.  However,  others  believed  that  architecture  would  be  highly 
beneficial  to  the  CRRA  analysis  and  were  especially  interested  in  developing 
architectures  that  could  show  relationships  as  well  as  redundancy.  These  practitioners 
theorized  that  showing  relationships  between  required  capabilities  and  the  capabilities 
provided  by  Air  Force  systems  and  services  could  expose  redundancies  and  gaps  (FAT, 
2008). 


Chapter  2  presented  how  architectural  products  are  used  to  assist  with  the 
construction  of  PSMs,  namely  the  use  of  an  OV-5  and  OV-6c,  as  well  as  an  OV-5  to 
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PSM  traceability  matrix  (Eller  2008: 190).  Chapter  2  also  presented  previous  work  that 
showed  the  development  of  an  OV-2  can  be  used  in  conjunction  with  the  OV-5  to 
provide  information  about  the  interconnectivity  and  interdependence  of  the  capabilities 
under  examination  (Eller,  2008:  26).  The  creation  of  these  models  is  still  essential  to  the 
ESM  process  as  they  tie  the  activities  under  examination  to  the  Air  Force  Concept  of 
Operations  (CONOPs)  and  ultimately  to  higher  level  DoD  guidance.  However,  by  using 
the  different  viewpoints  associated  with  the  Draft  DoDAF  2.0,  a  better  representation  of 
the  vital  information  can  be  developed,  ultimately  leading  to  a  better  analysis  of  the 
capabilities.  A  specific  example  of  this  is  the  new  Capability  Viewpoint,  which  offers  the 
ability  to  help  the  CRRA  stakeholders  understand  the  strategic  and  political  emphasis 
placed  on  a  requirement.  Given  there  are  many  different  capabilities  available  to  the 
decision  makers  that  potentially  offer  a  similar  end  result,  having  the  strategic  guidance 
available  will  enable  the  CRRA  practitioners  to  perform  the  best  possible  analysis  by 
only  considering  the  capabilities  relevant  to  the  scenario.  For  example,  if  the  end  goal  is 
to  eliminate  the  opposing  force’s  early  warning  system,  there  may  be  multiple  ways  to 
accomplish  the  mission;  however,  knowing  any  political  or  strategic  limitations  may 
reduce  the  available  options,  thereby  enabling  the  SMEs  to  only  focus  on  mission  areas 
and  systems  relevant  to  the  scenario.  A  capability  gap  may  be  emphasized  if  previously 
unknown  limitations  (such  as  troop  strength,  number  and  type  of  aircraft,  or  political 
limitations)  are  divulged  to  PSM  developers  and  the  SMEs  responsible  for  scoring  the 
PSMs.  For  these  reasons  it  is  important  to  provide  the  SMEs  with  more  scenario  specific 
information. 
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Based  on  what  this  research  was  able  to  uncover,  the  scoring  guidance,  to  include 
scenario  assumptions  and  limitations,  are  provided  to  SMEs  in  text  paragraph  form.  The 
SMEs  have  the  daunting  task  of  reading,  interpolating,  and  understanding  the 
ramifications  any  action  or  inaction  would  have  on  probability  of  success  while  putting 
the  scenario  in  context  with  the  current  and  future  world  climates.  Furthermore,  the  text 
appeared  to  lack  specific  explanations  of  what  resources  would  be  at  the  disposal  of  the 
Air  Force  in  the  scenario.  For  example,  a  probability  of  success  provided  under  the 
assumption  of  adequate  manpower  or  correct  configuration  of  airframes  may  not  be  an 
accurate  depiction  of  the  true  probability  of  success.  While  the  CRRA  scenarios  contain 
the  strategic  goals  of  that  scenario,  it  is  unclear  if  the  strategic  guidance  set  forth  in  the 
scenarios  approved  by  the  Office  of  the  Secretary  of  Defense  (OSD)  is  adequately 
explained  to,  or  understood  by,  the  CRRA  practitioners  or  SMEs.  Such  information 
should  be  considered  during  the  PSM  construction,  scoring  and  analysis  processes. 

The  Capability  Viewpoint  (CV),  can  communicate  various  types  of  data,  though 
normally  they  are  used  to  relate  high-level  strategic  information  and  guidance.  A  well- 
developed  CV  can  directly  state  overall  strategic  goals,  political  considerations, 
budgetary,  time  or  material  constraints  and  possibly  other  considerations  that  may  be 
applicable  to  the  given  scenario.  By  incorporating  Capability  Viewpoints  into  the  design 
and  development  of  the  PSMs  or  ESMs,  Stakeholders  can  ensure  that  strategic  guidance 
is  not  lost  between  the  approved  scenarios  and  the  PSM  or  ESM  scoring.  By  presenting 
the  developed  Capability  Viewpoints  to  the  SMEs  during  the  scoring  session,  SMEs  may 
help  uncover  any  hidden  capability  gaps  brought  out  by  the  strategic  guidance.  The 
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research  suggests  a  simplified  strategic  guidance  using  a  CV-1,  as  described  in  the  Draft 


DoDAF  v  2.0.  An  example  is  provided  in  Figure  4.1. 


Vision  (CV-1):  Imaginary  Country  Scenario 

Scenario:  Imaginary  Country  has  internal  political  opposition  from  the  flourishing  Authoritarian  Red 
Party  to  the  current  ruling  power  (Blue  Party).  The  United  States,  a  strong  ally  of  Imaginary 
Country  and  the  Blue  Party  leadership,  has  a  vested  interest  in  retaining  strong  economic  and 
political  ties  to  Imaginary  Country.  Provide  military  and  diplomatic  assistance  to  the  current 
ruling  party  in  an  effort  to  thwart  advances  of  the  Red  Party.  Due  to  security  measures,  use  of 
in-country  facilities  are  not  optimal,  nearest  bed  down  is  Another  Country,  807  nautical  miles 
due  south  of  Capitol  City,  Imaginary  Country. 

Objectives: 

1.  Retain  diplomatic  and  economic  ties  to  Blue  Party 

2.  Ensure  populace  not  harassed  into  submission  by  Red  Party 

3.  Retain  good  will  and  support  with  Imaginary  Country  population 

4.  Limit  damage  of  current  infrastructure,  reducing  threat  to  population  and  cost  to  rebuild 

5.  Retain  diplomatic  standing  with  current  and  future  allies 

6.  Limit  neighboring  country  strain  on  resources  by  influx  of  refugees  from  Imaginary  Country 

7.  Limit  increase  in  operations  tempo  to  125%  of  current  operations  tempo 

Authorizations:  36,000  military  personnel  (8,000  Air  Lorce,  Specific  ALSCs  &  Numbers  can  be  listed 
here) 

25  Aircraft  [A-10  (10);  B-1B  (3);  B-2  (2);  B-52  (1);  L-16  (8);  KC-135  (1)] 

Limitations: 

1.  Budgetary  constraints 

2.  Manpower  shortages  (330,000  members  -  #  currently  deployed/scheduled  for  deployment) 

3.  Time  constraints 

a.  Mobilization  time  (Limited  time  to  react  -  short  notice  «  96  hours) 

b.  Imaginary  Country  political  stabilization  time  (^24  months  after  height  of  conflict) 

c.  Timeline  for  withdrawal  (^4  years) 

4.  American  political  climate  (Military  conflict  unpopular) 

5.  World  standing 
Other  Considerations: 

1.  Effect  on  American  Trade 

_ 2.  Public  Support  for  another  contingency _ 

Figure  4.1  Capability  Viewpoint  (CV-1) 

As  with  any  scenario,  certain  capabilities  are  required  for  mission  success. 

During  the  research  it  was  discovered  that,  while  the  SMEs  were  asked  to  provide  P(S) 
for  many  nodes  given  an  identified  scenario,  they  were  not  given  adequate  information. 
For  example,  node  23.04  in  the  ACS  Homestation  Sustainment  PSM,  asked  SMEs  to  give 
a  P(S)  for  “Maintain  Law  and  Order”.  No  base  was  identified,  the  identified  base  was 
simply  identified  as  “base  X”,  instead  the  SMEs  were  given  suggested  bases  upon  which 
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they  should  model  their  score  and  rationale.  These  representative  bases  were  identified 
by  name,  but  no  statistics  were  detailed.  Of  course  they  are  different  in  location,  local 
populace,  base  population,  strength,  land  mass  and  security  requirements,  but  during  the 
scoring  it  is  assumed  that  any  P(S)  for  base  security  would  be  the  same.  In  the  ACS 
Smartbook,  the  SMEs  were  instructed  to  consider  between  60%  and  75%  of  the  base 
population  was  deployed  and  another  10%  to  14%  could  not  deploy  or  perform  normally 
(AF/A4LX,  2008:4-7).  Given  that  the  SMEs  were  not  informed  of  the  initial  base 
population,  or  the  remaining  quantity  or  quality  (skill  level)  of  forces,  it  would  be  almost 
impossible  for  a  SME  to  adequately  determine  the  level  of  ability  to  protect  a  military 
installation.  This  cannot  be  sufficiently  accomplished  without  knowledge  of  personnel 
strength  and  an  in-depth  knowledge  of  base-specific  security  requirements.  The 
assumptions  made  by  the  CRRA  practitioners,  concerning  the  undefined  supply  of  the 
appropriate  systems,  personnel  and  facilities  were  seen  as  questionable. 

If  a  CV-1  is  adequately  prepared  and  used  to  populate  Systems  Views  (SV),  the 
CRRA  practitioners  as  well  as  SMEs  can  use  these  architectures  to  rationalize  realistic 
P(S)  scores.  The  current  CRRA  process  literature  obtained  from  AF/A5X-C  includes 
mapping  systems  to  the  operational  activities  in  the  PSMs  (AF/A5X-C,  2007).  The 
completion  of  this  step  would  satisfy  the  input  requirements  for  generating  a  SV-5b.  By 
developing  SV-5  (either  a  or  b),  it  is  possible  to  see  where  there  are  system  redundancies. 
An  SV-5a  has  three  intended  functions  “tracing  functional  system  requirements  to  user 
requirements,  tracing  solution  options  to  requirements,  and  identification  of  overlaps” 
(DoD,  2008b:249).  Whereas  SV-5b  “maps  systems  requirements  to  user  requirements, 
traces  solution  options  to  requirements  and  identifies  overlaps”  (DoD,  2008b:250).  As 
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the  research  mentioned  earlier,  it  was  desired  among  the  CRRA  stakeholders  to  show 
redundancies.  Any  redundancies  could  be  thoroughly  evaluated  to  decide  if  they  were 
necessary  or  unnecessary.  Unnecessary  redundancies  could  then  be  offered  as  trade 
space,  while  any  gaps  identified  through  this  process  could  be  analyzed  to  reveal  the 
extent  of  any  deficiencies.  Additionally,  this  could  be  used  as  part  of  the  ongoing  CRRA 
Potential  Areas  to  Accept  Risk  (PAAR)  analysis  discussed  in  Chapter  2.  Furthermore, 
any  operational  activities  that  do  not  have  a  system  mapped  to  it  can  be  examined  to 
determine  by  what  means  that  activity  is  accomplished,  potentially  uncovering  a 
capability  gap. 

From  a  well  developed  SV-5  that  lists  all  the  systems  used  to  accomplish  the 
operational  activities,  the  CRRA  practitioners  should  develop  a  SV-7  (Systems  Measures 
Matrix)  that  defines  performance  characteristics  and  measures  for  the  systems  in  use.  By 
developing  a  SV-7,  CRRA  practitioners  as  well  as  SMEs  can  use  their  expert  knowledge 
to  understand  what  systems  the  scenario  contains,  and  the  associated  performance  metrics 
of  those  systems.  Using  the  advanced  scenario  data  provided  will  enhance  the  accuracy 
of  any  P(S)  given  by  the  SME  while  also  giving  the  SME  defendable  rationale  for  the 
scores  they  provide.  Additionally,  the  CRRA  practitioners  can  see  and  understand 
functional  redundancies  as  well  associated  gaps  between  the  systems.  The  following  SV- 
7  in  Table  4.1  was  populated  with  both  type  and  number  of  aircraft,  along  with  the 
performance  measures  of  the  aircraft,  for  operation  in  the  Imaginary  Country  scenario. 
Aircraft  and  their  capabilities  are  presented  here  as  an  example;  however,  this  can  be 
adapted  to  describe  any  type  of  system  or  service  and  the  performance  characteristics 
they  provide  or  contribute.  Using  this  viewpoint,  numbers,  types  and  configurations  of 
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aircraft  (for  example)  can  be  modified  to  provide  the  best  combinations  using  the  least 


amount  of  resources. 


Table  4.1.  Systems  Measures  Matrix  (SV-7) 


Systems  Measures  Matrix  (SV-7):  Imaginary  Country  Scenario 

Number  of 
Aircraft: 

10 

3 

2 

1 

8 

0 

1 

Type  of  Aircraft: 

A-10 

B-1B 

B-2 

B-52 

F-16 

F-22A 

KC-135 

Range 

695 

nm 

6,478 

nm 

6,000 

nm 

7,652 

nm 

1,740 

nm 

1,600 

nm 

1,500 

nm 

Cannon 

30  mm 

20  mm 

20  mm 

Ground  Support 

Yes 

Yes 

Yes 

Nuclear 

Weapons 

No1 

Yes 

Yes 

Mixed 

Ordinance 

16,000 

lbs 

75,000 

lbs 

50,000 

lbs 

70,000 

lbs 

20,450 

lbs 

Unknown 

lbs 

Naval  Support 
(mines) 

Yes 

Yes 

Missiles 

Yes 

Yes2 

Yes 

Rockets 

Yes 

Yes 

Transferable  fuel 
load: 

200,000 

lbs 

1  START  Treaty  Requirement  (Air  Force  Fact  Sheet,  2008) 

2  Air-to-air,  Air  to  ground,  Anti-Ship 

Items  in  Red,  may  not  meet  minimum 
capability  requirements  as  defined  in  CV-1 


As  with  any  form  of  communication,  care  must  be  taken  to  not  overload  these 
viewpoints  with  superfluous  information.  These  viewpoints  are  meant  to  be  a  simplified 
reference  so  decision  makers  can  base  their  scores  on  set  parameters  that  may  differ 
between  scenarios. 

These  suggestions  are  not  intended  to  indicate  an  exhaustive  list,  or  lead  the 
reader  to  the  conclusion  that  these  are  the  only  applicable  views  useful  for  CRRA 
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analysis;  a  detailed  evaluation  of  the  draft  DoDAF  v  2.0  and  its  associated  viewpoints 
would  be  beneficial  to  the  CRRA  process. 

A  common  complaint  among  many  of  the  CRRA  practitioners  concerned  the  lack 
of  centralized  software  that  was  core  to  the  development,  planning  and  scoring  of  the 
CRRA  process.  Currently,  there  is  no  single  software  that  holds  the  entire  architecture 
utilized  by  the  CRRA.  A  combination  of  many  types  of  software  is  used  to  accomplish 
different  parts  of  the  CRRA  process.  Software  products  such  as  Provision,  Word,  Excel, 
Visio  and  Power  Point,  to  name  a  few,  are  used  to  input,  manipulate,  store  and  produce 
the  desired  output.  Integration  of  DoDAF  v  2.0  compliant  architecture  is  important  to 
adequately  develop  and  score  the  CRRA.  DoDAF  v  2.0  compliant  software  should  be 
purchased  or  developed  to  provide  a  central  repository  for  AF  architecture  used  by  the 
CRRA,  to  include  the  scoring  and  analysis  portions.  By  having  a  central  repository, 
development,  manipulation  and  analysis  of  the  architecture  could  be  readily  accessed  and 
further  analyzed  iteratively  and  continuously. 

There  are  two  types  of  architectural  analysis,  static  and  dynamic.  Static  analysis 
is  based  on  data  which  has  been  extracted  from  the  architecture,  such  as  historical  data 
used  to  develop  future  trends.  This  usually  uses  simpler  tools  such  as  visual 
comparisons.  Conversely,  Dynamic  analysis  is  “running”  a  version  of  architecture  which 
has  been  developed  as  an  executable  model  to  evaluate  performance  or  run  multiple 
variables  in  different  situations  (DoD,  2008a: 83).  The  viewpoints  suggested  have  been 
static  architecture.  ESMs,  which  are  discussed  in  the  next  section,  are  a  type  of 
executable  dynamic  architecture  that  has  been  specifically  developed  to  analyze  the 
CRRA. 
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4.3  Modeling  capabilities  using  ESMs 

This  section  will  present  a  technique  called  Extended  Sequence  Models  (ESM) 
that  allows  the  CRRA  practitioners  to  uncover  and  examine  the  dependencies  of  activities 
within  the  modeled  system.  Additionally,  the  technique  can  account  for  sufficiency  of 
capability  or  quantity  of  assets  to  examine  how  it  affects  operational  risk  while  more 
formally  identifying  what  nodes  represent  activities  that  are  part  of  a  process  and  what 
nodes  represent  environmental  conditions  of  the  scenario.  Current  PSMs  cannot  account 
for  any  of  these  conditions.  While  this  research  aims  to  suggest  methods  that  are  useable, 
the  usability  of  the  ESM  techniques  will  have  to  be  determined  by  the  CRRA 
practitioners. 

This  section  is  presented  as  a  methodology  the  CRRA  practitioner  can  follow  to 


successfully  perform  the  technique.  Figure  4.2  gives  an  overview  of  the  method  that  will 
be  described. 


Figure  4.2  ESM  Implementation 
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Step  0  of  this  process  has  already  been  described  in  the  previous  section.  The 
creation  of  architectural  products  to  adequately  prepare  the  SMEs  to  provide  the  most 
accurate  input  possible  is  a  critical  preparatory  step.  The  ESM  methodology  continues  to 
the  construction  of  the  model  framework.  This  can  be  accomplished  by  a  separate  group 
of  SMEs  from  the  scoring  SMEs  or  it  could  be  the  same  group  of  experts  if  the  CRRA 
practitioners  so  decide. 

Step  1  :  The  first  step  to  creating  the  framework  is  to  identify  all  activities  that 
must  be  accomplished  for  the  successful  completion  of  the  mission  under  examination  or 
the  successful  demonstration  of  the  capability.  This  step  is  accomplished  in  the  current 
method  and  is  supported  by  an  adequate  architecture’s  OV-5  and  OV-6c. 

Step  2:  Environmental  factors  are  then  identified.  Environmental  factors  are 
different  from  activities  in  that  they  are  not  performed.  They  are  conditions  of  the 
environment  the  activities  are  being  performed  in.  Examples  of  environmental  factors 
might  be  the  security  environment  measured  by  attacks  per  day  or  weather  conditions 
such  as  visibility  in  miles. 

Step  3:  Each  activity  and  environmental  condition  should  be  evaluated  to 
determine  the  metric  that  will  be  used  to  describe  it.  Currently,  every  activity  is 
measured  by  probability  of  success.  That  might  remain  the  dominant  metric,  however, 
tasks  that  don’t  clearly  fit  the  success  or  failure  assumption  can  be  assigned  another 
metric.  For  example,  a  training  activity  can  have  the  measurement  Number  of  Airmen 
Trained  or  a  supply  activity  could  measure  Number  of  Parts  Ordered.  The  metric 
doesn’t  even  have  to  be  one  of  quantity;  it  could  be  one  of  quality.  For  example,  an 
activity  titled  Establish  communication  link  can  have  a  metric  Percent  of  time 
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communication  link  available.  The  metric  should  be  chosen  based  on  two  criteria.  The 


first  is  whether  it  will  likely  affect  the  successful  completion  of  the  mission  or  capability. 
The  second  is  whether  the  CRRA  practitioners  determine  examining  that  metric  is 
valuable. 

Step  4:  Once  all  activities  and  environmental  factors  are  identified  and  assigned  a 
measurement  variable,  they  are  divided  into  two  categories.  The  first  group  contains 
those  nodes  that  have  an  outcome  that  other  activities  depend  on;  this  group  will  be  called 
the  ‘controlling’  nodes.  Controlling  nodes  can  have  any  type  of  measurement  variable. 
The  outcomes  of  the  controlling  nodes  contribute  to  the  overall  probability  of  success  of 
the  mission  only  indirectly  through  their  affect  on  other  nodes.  The  second  group 
contains  those  nodes  that  have  an  outcome  that  doesn’t  affect  other  nodes;  the  outcomes 
of  these  nodes  directly  affect  the  successful  completion  of  the  overall  mission.  This 
group  will  be  called  ‘non-controlling’  nodes  and  can  only  contain  nodes  whose 
measurement  variable  is  probability  of  success.  This  is  obviously  a  subjective  process. 
With  some  careful  thought,  almost  every  activity  can  result  in  performance  differences  in 
other  activities.  The  goal  is  to  find  the  balance  that  allows  more  dependencies  to  be 
uncovered  than  the  current  method  allows,  while  keeping  the  technique  usable  and 
accessible  to  the  CRRA  community.  The  increase  of  represented  dependencies  greatly 
increases  the  number  of  scores  SMEs  are  required  to  provide,  as  will  become  apparent  in 
the  following  paragraphs. 

Step  5:  The  controlling  nodes  need  to  be  linked  to  the  nodes  they  influence.  The 
graphical  way  to  represent  this  link  is  to  draw  a  dotted  arrow  from  the  controlling  node  to 
the  node  that  is  depending  on  it.  This  is  consistent  with  the  Unified  Modeling  Language 
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(UML)  technique  to  show  a  dependency  relationship  (Larman,  2002:295).  Controlling 
nodes  can  be  linked  to  other  controlling  nodes  creating  chains  of  controlling  nodes; 
however,  eventually  they  need  to  link  to  a  non-controlling  node  whose  metric  is 
probability  of  success.  The  controlling  nodes  that  are  intermediary  nodes  in  the  chain  can 
have  any  type  of  metric  just  like  any  other  controlling  node.  Additionally,  it  is  acceptable 
to  have  non-controlling  nodes  that  have  no  dependencies. 

The  construction  of  the  model  might  require  controlling  node  variables  to  be 
combined  in  a  way  other  than  through  dependency  relationships.  To  handle  this  situation 
a  Function  Node  can  be  inserted  with  the  appropriate  logic  assigned  to  it.  The  Function 
Node  transforms  input  variables,  provided  by  controlling  nodes  (graphically  shown  by 
solid  arrows),  through  a  function  to  an  output  required  by  other  nodes,  an  example  of  a 
function  node  is  presented  in  Figure  4.3.  A  complete  list  of  model  components,  their 
symbols,  and  definitions  is  presented  in  Appendix  A. 


Figure  4.3:  Example  of  a  Function  Node 


55 


Step  6:  At  this  point  the  model’s  framework  has  been  constructed.  The  next  step 
is  populating  the  nodes  with  probability  density  functions  provided  by  the  scoring  SMEs. 
The  SMEs  will  start  with  activities  that  are  not  dependent  on  any  other  activity;  these  are 
the  independent  nodes.  If  the  activity’s  measurement  variable  is  probability  of  success, 
the  SME  will  "score"  the  node  identical  to  the  current  method  by  providing  a  most  likely, 
worst  case,  and  best  case  value.  Activities  that  have  measurement  variables  other  than 
probability  of  success  will  be  handled  very  similarly.  The  SME’s  will  determine  the 
most  likely  value  of  that  variable,  the  greatest  value  the  variable  would  attain,  and  the 
lowest  value  that  the  variable  would  likely  attain,  given  the  scenario. 

Step  7:  Following  the  scoring  of  the  independent  nodes  SME’s  attention  turns  to 
the  dependent  nodes.  Dependent  nodes  are  those  nodes  whose  probability  density 
functions  depend  on  the  outcome  of  one  or  more  controlling  nodes.  The  SMEs  provide 
scores  for  the  dependent  node  based  on  the  value  of  the  controlling  nodes.  For  example, 
SMEs  decide  the  probability  of  successfully  on-loading  cargo  and  passengers  to  an  airlift 
mission  in  the  time  required  depends  on  the  amount  of  passengers  and  cargo  to  be  loaded. 
Based  on  the  scenario  and  the  airframe  in  question  SMEs  determine,  in  step  6,  the  range 
of  passengers  could  be  between  0  and  92  while  the  range  of  cargo  is  likely  to  be  in  the 
range  of  10  to  20  tons  with  some  most  likely  value  for  passengers  and  cargo  as  shown  in 
Figure  4.4. 
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Figure  4.4  ESM  Example 

SME’s  then  provide  the  probability  of  successfully  on-loading  cargo  and  passengers 
based  on  the  extremes  of  the  controlling  nodes.  If  a  dependent  node  is  dependent  on  n 
controlling  nodes  the  SMEs  need  to  provide  at  least  (n+1)  scores  to  fully  describe  what 
the  response  will  be  to  the  dependent  node.  Each  of  the  scores  should  correspond  to  a 
unique  combination  of  values  for  the  controlling  nodes.  For  the  example  in  Figure  4.4, 
the  dependent  node  loading  Cargo/Passengers  is  dependent  on  two  nodes,  therefore  the 
SMEs  have  to  provide  at  least  three  probabilities.  For  this  example,  the  SMEs  determine 
the  most  likely  probability  of  success  for  loading  the  cargo  and  passengers  is  0.97  given 
there  are  0  passengers  and  10  tons  of  cargo.  Similarly,  the  SMEs  determine  the  most 
likely  probability  of  success  is  0.90  given  there  are  92  passengers  and  20  tons  of  cargo 
and  they  provide  a  most  likely  probability  of  success  of  0.93  given  0  passengers  and  20 
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tons  of  cargo.  The  SMEs  also  provide  an  upper  and  lower  bound  around  the  score  they 
provide.  This  creates  the  triangular  pdf  and  represents  the  uncertainty  they  have  in  the 
score  they  provided.  These  SME  responses  are  depicted  in  Figure  4.5. 


Figure  4.5  Most  likely  Probability  value  dependence  on  two  controlling  nodes 
The  technique  presented  above  results  in  the  minimum  data  collection  needed  to 
complete  the  model.  More  specifically,  it  requires  (n+1 )  scores  from  the  SMEs  for  each 
node  where  n  is  the  number  of  controlling  nodes.  By  mandating  that  the  probabilities 
provided  by  the  SMEs  are  based  on  the  extremes  of  the  possible  variable  ranges  of  the 
controlling  nodes,  this  model  guarantees  linear  independence  of  the  controlling  node 
samples  and  prevents  an  underdetermined  situation  that  would  not  result  in  a  dependency 
equation.  However,  the  technique  can  be  expanded  to  accept  an  increase  in  data  points. 
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Instead  of  only  looking  at  the  extremes  of  the  controlling  node  variables,  SMEs  can 
provide  probabilities  of  success  for  points  somewhere  between  the  extremes  of  the 
independent  variable  range.  Additionally,  for  the  same  input  variables  multiple  SMEs 
could  provide  differing  probability  of  success  that  will  all  be  taken  into  consideration 
when  approximating  the  relationship  in  step  8. 

While  obtaining  more  data  points,  care  must  still  be  taken  to  ensure  the  minimum 
requirements  are  met  as  to  not  create  an  underdetermined  situation.  This  can  be 
accomplished  by  the  scoring  facilitator  choosing  the  values  of  the  controlling  nodes  that 
the  SMEs  will  use  to  determine  the  score  they  assign.  Any  scores  within  the  range  of 
possible  values  of  the  controlling  nodes  can  be  selected.  The  scoring  facilitator  needs 
only  ensure  the  rank  of  the  input  variable  matrix  X,  is  equal  to  or  greater  than  (n+1) 
where  n  is  the  number  of  controlling  nodes.  The  minimal  method  presented  earlier 
ensured  the  rank  equaled  (n+1)  by  selecting  the  lowest  value  for  all  the  variables,  the 
highest  value  for  all  the  variables,  then  a  mix  of  highest  and  lowest  until  the  minimum 
number  of  scores  had  been  collected  from  the  SMEs.  One  way  to  guarantee  the  rank  is 
great  enough  is  to  follow  the  minimal  method  presented  above  by  selecting  the  extremes 
of  the  variable  ranges.  Only  after  that  method  is  accomplished  the  scoring  facilitator  can 
start  choosing  values  for  the  controlling  nodes  that  are  somewhere  in  between  the 
extremes.  This  ensures  the  problem  is  over  determined  and  a  least  squares  fit  can  be 
accomplished  to  find  the  linear  regression  (Johnson,  1993:  207). 

Step  8:  The  relationship  between  the  controlling  nodes  and  the  nodes  they  control 
is  then  approximated  based  on  the  input  provided  by  the  SMEs  in  step  7.  For  the  general 
case  we  are  trying  to  find  a  linear  approximation  to  the  relationship  that  takes  the  form: 
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Pi(s)  =  Pi, 0  +  Pi,  A  +  Pi, 2%2  ■+ - F  Pi,nXn 

Where,  Pj(s),  is  the  probability  of  success  for  node  i,  X/,  is  the  value  of  the  first 
controlling  node,  X2  is  the  value  of  the  second  controlling  node,  and  Xn  is  the  value  of  the 
nth  controlling  node.  The  /?  values  are  the  equation  parameters  that  control  the  weight  or 
strength  each  controlling  node  has  on  the  final  probability  where  /?j  n  is  the  nth  parameter 
for  the  ith  node. 

For  the  example  shown  in  Figure  4.5,  the  relationship  between  probability  of 
successfully  on-loading  cargo  and  passengers  based  on  the  quantities  of  passengers  and 
cargo  to  be  loaded  is  described  by  the  equation: 

pi(.s )  =  Pi,o  +  Pi,  1^1  +  Pi, 2^2  (7) 

Where  Xi  and  X2  are  the  values  of  the  controlling  nodes  (Xi=  Number  of  passengers, 
X2=tons  of  cargo). 

SME’s  are  asked  to  provide  probability  of  success  scores  based  on  various  values 
of  the  controlling  nodes.  This  yields  a  system  of  linear  equations: 

Pi,l(.s)  =  Pi,0  +  Pi, 1X1,1  +  Pi, 2X2,1  4 - f  Pi.nXn.l 

Pi,2(s)  =  Pi, 0  +  Pi, 1X1, 2  +  Pi, 2X2,2  4 - f  Pi,nXn, 2 

Pi, 3^s)  =  Pi,0  4"  Pi, 1X1, 3  +  Pi, 2X2, 3  4 - f  Pi,nXn, 3 

Pi,m(s )  =  Pi,0  4"  Pi,lXi,m  +  Pi, 2^2, m  4 - F  Pi,nXn,m 

Where  Pi,m(s)  is  the  ith  node’s  score  for  the  mth  trial  while  Xnm  is  the  nth  controlling 
node’s  value  for  the  mh  trial. 

For  the  case  where  exactly  (n+1)  scores  have  been  provided  by  the  SMEs  to 
define  a  relationship  between  a  dependent  node  and  n  controlling  nodes  with  n 
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dependent  relationships,  the  SME  responses  yield  a  system  of  (n+1 )  linear  equations. 

The  equations  for  the  example  in  Figure  4.5  are: 

0.97  =  /?10  +  /?i,i(0)  +  /?i,2(10) 

0.90  =  /?!  o  +  /?i,i(92)  +  /?!  2(20) 

0.93  =  jff1,0  +  jff1,1(0)  +  ^1>2(20) 

The  values  of  (310,  ptland  /?12  that  satisfy  the  equations  need  to  be  found.  A  simple  way 
to  find  these  values  is  by  using  matrices  where  (Neter,  1996:227) 
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P  = 


r/?oi 

Pi 


Then:  Y  =  X/3 


P  =  X-'Y  (8) 

The  technique  used  to  find  the  values  of  X  and  Y  guarantees  X  will  be  a  square  matrix 
with  linearly  independent  rows  in  X.  This  allows  us  to  perform  equation  8  without 
concern  whether  X  is  invertible  or  that  X  and  Y  matrices  will  not  multiply  (Johnson, 

1993:  54,96). 

The  example  would  produce: 


7V 

( 

T 

0 

10' 

V1 

0.9T 

'  1.01  ‘ 

Pi 

= 

1 

92 

20 

0.90 

= 
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.  -0.004. 

We  can  now  update  Equation  7  for  this  node  with: 
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Probability  of  Success  =  1.01  +  (— 0.0003)^  +  (— 0.004)A2 
This  is  scalable  to  any  number  of  controlling  nodes;  however  each  new  controlling  node 
requires  the  SMEs  provide  another  probability  of  success  relationship. 

To  handle  the  situation  where  the  SMEs  provide  some  number  of  scores  greater 
than  (n+l),a  linear  regression  is  computed  using  the  same  least  squares  technique  above 
however  using  the  generalized  equation.  Once  the  matrices  of  Y  and  X  are  developed, 
the  equation  parameters,  /?  must  be  found  such  that  ||T  —  Xfi\\  is  minimized.  Equation  8, 
which  is  presented  again  below,  was  used  in  the  minimal  data  point  method  to  find  the 
equation  parameters,  /?.  This  equation  cannot  be  used  for  the  over-constrained  case  with 
greater  data  points  because  we  cannot  guarantee  that  X  is  square  or  invertible.  This 
causes  equation  8  to  be  undefined.  Therefore,  the  pseudo  inverse  is  used  to  find  the 
generalized  equation  for  /?  presented  as  equation  9  (Neter,  1996:227). 

p  =  X~xY 

P  =  (X'X)  (9) 

As  mentioned  above,  because  the  expanded  data  collection  method  allows  for  a 
large  number  of  SME  scores,  the  CRRA  practitioner  can  also  decide  to  have  multiple 
SMEs  determine  their  own  scores.  Different  SMEs  will  undoubtedly  provide  slightly 
different  probability  scores  given  the  same  group  of  input.  This  method  creates  a  best  fit 
relationship  thus  aggregating  their  inputs  to  arrive  at  a  single  linear  model. 

It  might  be  decided  in  future  iterations  of  the  CRRA  that  obtaining  multiple  data 
points  from  many  SMEs  is  more  desirable  than  the  current  method  of  getting  consensus 
on  one  score.  This  could  prevent  a  group  think  mentality  where  an  inaccurate  score  is 
presented  based  on  the  dynamics  and  pressures  within  the  group.  Another  benefit  to 
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using  this  expanded  method  is  it  can  be  less  time  consuming  than  getting  agreement  on  a 
minimal  set  of  scores  from  many  people.  Additionally,  more  data  points  could  be 
acquired  from  individuals  who  are  not  able  to  attend  the  scoring  session  in  person.  It  is 
possible  that  tens  or  even  hundreds  of  individuals  across  the  Air  Force,  from  Airmen 
working  on  the  flight  line  to  Wing  Commanders,  could  provide  their  perspective  on  a 
node  they  have  experience  with  to  provide  a  more  complete  understanding  of  the 
dependency  relationships. 

The  example  above  shown  in  Figure  4.5,  was  used  to  find  the  ‘most  likely’  or 
mode  of  the  pdf  for  on-loading  the  passengers  and  cargo.  To  complete  the  triangular  pdf, 
an  upper  and  lower  bound  must  be  determined.  The  technique  of  approximating  a  linear 
relationship  could  be  repeated  two  additional  times  to  find  the  response  on  the  highest 
and  lowest  bounds  based  on  controlling  node  values.  This  would  allow  the  width  of  the 
triangular  distribution  to  change  based  on  the  controlling  variables;  however,  this  would 
also  require  additional  input  from  SMEs  and  additional  calculations  to  execute  the  model 
which  requires  additional  resources  to  complete  the  analysis.  Therefore,  assuming  the 
bounds  of  the  probability  density  function  respond  identical  to  the  ‘most  likely’  value  is 
appropriate.  The  SMEs  would  determine  an  upper  and  lower  bound  around  the  highest 
possible  value  for  the  mode.  The  width  of  that  range  would  stay  constant  as  the  most 
likely  value  shifts.  In  the  example  of  Figure  4.5,  the  SMEs  determined  0  passengers  and 
10  tons  of  cargo  would  result  in  a  most  likely  probability  of  success  of  0.97.  The  SMEs 
would  then  provide  an  upper  and  lower  bound.  If  the  SMEs  determined  the  bounds  were 
0.95  and  0.99,  then  as  other  combinations  of  input  variables  creates  different  most  likely 
probability  distributions  the  upper  and  lower  bounds  would  remain  0.02  above  and  below 
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the  mode.  For  example,  if  there  were  45  passengers  and  13  tons  of  cargo  the  model 
would  produce  a  most  likely  probability  of  0.945.  The  lower  bound  would  be  0.965  and 
the  upper  bound  would  be  0.925. 

Step  9:  Similar  to  the  current  PSM  approach,  a  Monte  Carlo  analysis  is 
performed.  It  starts  by  selecting  a  value  for  all  independent  nodes  from  the  triangular 
distributions  provided.  In  the  generic  model  presented  in  Figure  4.6,  values  are  selected 
for  nodes  3,5,  and  6.  Once  the  values  are  selected  the  probability  distributions  are 
defined  for  nodes  influence  by  those  outcomes.  The  probability  density  function  for  node 
2  and  node  4  would  be  found  based  on  the  outcome  of  nodes  5  and  6  respectively.  At 
that  point,  the  simulation  selects  values  for  those  nodes  whose  probability  distribution  has 
been  calculated  (nodes  2  and  4  below). 

This  continues  to  roll  up  until  all  non-controlling  nodes  (nodes  1,  2,  and  3)  have  a 
value  selected.  Non-controlling  nodes  must  have  a  measurement  variable  of  probability 
of  success;  therefore,  the  overall  probability  of  success  for  the  mission  is  found  by 
evaluating  the  probabilities  of  the  non-controlling  nodes.  If  the  nodes  are  presented  in  a 
series,  all  nodes  must  be  successful  for  the  entire  mission  to  complete  successfully.  To 
find  the  total  probability,  the  probabilities  of  the  nodes  are  simply  multiplied  to  find  the 
total  system  probability;  however,  it  is  also  possible  to  include  redundant  nodes.  In  this 
case,  the  total  probability  would  be  calculated  using  equation  2  and  equation  3.  Shown 
again  here: 

P(.S) Total  =  +  Pc(S)  -  PB<S)PC<S) 

P(S)ro,al  =  (1  -  n?,i(l  -  P(S) ,))  (10) 
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Non-controlling  nodes  in  an  ESM  don’t  necessarily  have  to  be  arranged  as  a  process. 

This  is  because  each  of  the  non-controlling  nodes  could  represent  an  activity  that  is 
ongoing  throughout  the  course  of  the  evaluation  period.  Either  solid  lines  or  branched 
lines  are  drawn  between  the  non-controlling  nodes  to  represent  whether  the  total 
reliability  is  calculated  using  the  series  approach  of  multiplying  all  the  probabilities  or 
calculated  as  redundant  nodes  using  equation  10.  Both  types  of  non-controlling  node 
connectors  can  be  seen  in  Appendix  A.  The  example  in  Figure  4.6  shows  a  series 
configuration. 

Additionally,  the  diamond  shaped  decision  nodes  are  handled  in  the  same  way  as 
the  current  PSM  method  as  explained  in  equation  4,  although  in  this  method  they  are  not 
called  decision  nodes.  For  use  in  PSMs,  as  well  as  ESMs,  the  decision  nodes  rarely 
represent  decisions;  they  more  often  represent  the  percent  of  time  various  nodes  are 
involved  or  not  based  on  the  conditions  of  the  scenario.  The  diamond  shaped  nodes 
determine  how  often  a  node  is  active.  Therefore,  they  have  been  renamed  Activating 
nodes  as  they  activate  other  nodes  some  fraction  of  the  time.  The  final  probability 
equation  for  Figure  4.6  is: 

P(S)Total  =  P1(S)[A(No)P2(S )  +  A(Yes)P3(S )]  (11) 
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Figure  4.6  ESM  Generic  Example 


Step  10:  Section  2.3  described  the  sensitivity  analyses  that  are  performed  on 
PSMs  to  determine  which  activities’  outcomes  have  the  greatest  impact  on  the  overall 
mission  probability,  and  ultimately  the  mission  consequence  and  risk.  This  is  a  crucial 
step,  possibly  more  important  than  finding  overall  mission  probability,  because  it  assists 
with  deciding  what  areas  need  additional  resources  in  the  future.  The  current  method 
accomplishes  this  sensitivity  analysis  using  two  methods,  availability  and  incremental. 
ESMs  accomplish  these  two  analyses  in  the  same  way. 

To  accomplish  the  availability  method,  all  nodes  are  kept  at  their  most  likely 
value  except  for  the  node  under  examination.  If  the  node  has  a  measurement  variable  of 
probability  of  success  its  score  is  increased  by  half  the  difference  between  its  most  likely 
value  and  1.0.  This  is  identical  to  the  current  technique  (FAT,  2008).  If  the 
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measurement  variable  is  not  probability  of  success,  the  measurement  variable  is  evaluated 
at  its  least  value  and  its  greatest  value  to  determine  the  difference  in  overall  risk. 

The  incremental  technique  is  accomplished  identically  to  the  current  technique. 
The  equation  that  describes  the  probability  of  success  for  the  entire  mission  is  identified, 
and  partial  derivatives  of  that  equation  with  respect  to  individual  nodes,  are  used  to 
determine  which  node  has  the  greatest  associated  sensitivity.  It  was  presented  in  equation 
1 1  above  that  the  overall  probability  of  the  model  in  Figure  4.6  is: 

Pivotal  =  P1(S)[A(No)P2(S)  +  A(Yes)P3(S)] 

We  can  further  expand  this  equation  by  recognizing  that 

Pi(s )  =  ft,0  +  Pi, MS)  +  PiMS) 

P4(S)  =  04>o  +  04l6ft(S) 

P2(S)  =  p2,o  +  p2MS) 

Where  a  and  {  represent  the  equation  parameters  for  the  other  two  nodes. 

Inserting  all  of  these  into  equation  1 1  results  in: 

P(S)Total  =  (01>O  +  ft, 4(/?4,0  +  04,6ft(ft»  +  ft,5P5(S))(A(No)(  02,0  +  02,5ft  (5)) 

+  A(Yes)P3(S)) 

This  equation  shows  the  relationship  of  the  total  probability  of  success  based  on 
the  outcome  of  the  lowest  level  nodes.  Each  partial  derivative  may  be  found  with  respect 
to  each  node.  The  node  whose  partial  derivative  has  the  greatest  magnitude  is  the  driving 
node.  While  this  example  shows  the  sensitivity  analysis  done  at  the  lowest  level  it  could 
be  accomplished  at  any  level.  For  example,  the  sensitivity  analysis  could  have  been 
accomplished  at  the  top  level  represented  by  the  equation  1 1  to  determine  which  of  the 
top  level  nodes  have  the  greatest  influence  on  the  outcome. 
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4.4  ESM  Assumptions 

In  addition  to  assuming  a  linear  relationship  between  controlling  and  non¬ 
controlling  nodes,  the  ESM  method  also  assumes  that  the  only  dependencies  between 
nodes  are  the  ones  presented  in  the  model;  the  nodes  are  independent  from  each  other  in 
all  other  ways.  The  current  method  assumes  all  nodes  are  independent  from  each  other. 
As  explained  earlier,  with  some  careful  thought  dependencies  can  be  found  between 
many  nodes;  this  new  method  allows  the  CRRA  practitioner  to  identify  the  dependencies 
that  could  be  important  to  the  analysis  and  examine  them  while  continuing  to  assume 
independence  with  other  nodes. 

ESMs  assume  that  the  dependency  relationships  can  be  modeled  as  linear.  In 
reality,  the  relationship  between  two  variables  might  not  be  linear.  By  making  this 
assumption,  ESMs  attempt  to  identify  a  first  order  approximation  and  builds  on  the 
current  PSMs  that  don’t  allow  any  dependencies. 

It  is  important  to  recognize  this  model  does  not  explicitly  show  a  process,  even  if 
the  activities  in  the  ESM  are  accomplished  within  a  process.  It  shows  the  activities  that 
must  be  completed  to  successfully  complete  the  mission  as  well  as  factors  dependent  on 
those  activities.  Some  or  all  of  these  activities  might  occur  simultaneously  or  they  might 
not.  The  ordering  of  some  activities  can  be  inferred  when  the  performance  of  an  activity 
is  dependent  on  the  outcome  of  another  discrete  activity.  This  is  a  byproduct  of  showing 
dependencies.  Therefore,  in  missions  where  there  are  discrete  activities  accomplished  in 
a  process  and  timing  information  is  of  importance  to  the  CRRA  practitioner,  it  would  be 
important  to  construct  the  ESM  to  show  the  ordered  accomplishment  of  activities  while 
also  showing  the  dependency  relationships  activities  have  on  each  other.  Even  in  such  a 
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case  the  probability  of  success  of  activities  can  still  be  related  to  environmental  factors 
using  the  techniques  described  in  this  new  method  without  any  change  to  the  technique 
described. 

Another  assumption  is  that  the  only  effect  controlling  nodes  have  on  the  overall 
probability  of  success  is  indirectly  through  their  influence  on  the  non-controlling  nodes. 
This  assumption  is  valid  because  in  the  case  a  controlling  node  does  directly  impact  the 
probability  of  success  of  the  overall  mission,  a  new  non-controlling  node  can  be  added 
that  is  dependent  on  the  controlling  node  in  question  and  captures  the  direct  influence  that 
was  identified.  In  the  example  of  Figure  4.4,  the  tonnage  of  cargo  impacted  the 
probability  of  successfully  loading  the  cargo  and  passengers  in  a  timely  manner.  If  it  is 
decided  that  the  tonnage  of  cargo  also  has  a  direct  affect  on  the  overall  probability  of 
success  of  the  mission,  e.g.  the  lower  the  tonnage  the  less  likely  the  mission  will  succeed, 
then  a  new  non-controlling  node  could  be  added  called  Enough  cargo  to  complete  the 
mission,  and  its  probability  of  success  can  be  linked  to  the  tonnage  of  cargo  controlling 
node.  This  way,  while  the  tonnage  drops  and  increases  the  probability  of  loading  the 
cargo  and  passengers,  it  decreases  the  probability  of  having  enough  cargo  to  complete  the 
mission. 

4.5  Verification  of  the  Model 

For  the  purpose  of  gaining  a  better  understanding  of  how  this  model  and  its 
supporting  method  would  be  implemented,  a  test  of  the  model  was  conducted.  A  small 
portion  of  the  2009  ACS  Homestation  Readiness  PSM  was  selected  for  the  test.  A 
review  of  this  PSM  resulted  in  the  identification  of  several  nodes  that  lent  themselves  to 
the  exploration  of  how  sufficiency  or  quantity  can  be  captured  and  dependencies  shown. 
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The  nodes  selected  were  Identifying  Surge  Requirements  (3.01),  Recall  Selected/Ready 
Reserve  (3.03),  and  Recall  Standby/Retired  Reserve/IRR  (3.05).  It  was  decided  that 
determining  how  many  Airmen  are  needed  and  how  many  Airmen  are  obtained  through 
the  reserves  could  shed  more  light  on  the  probability  of  completing  the  mission  than 
simply  identifying  the  likelihood  of  successfully  identifying  the  number  of  Airmen 
required  and  the  likelihood  of  successfully  filling  that  number. 

The  logic  of  the  current  PSM  method  broke  down  when  applied  to  the  Recall 
Standby/Retired  Reserve.  The  current  method  calls  for  each  node  to  be  scored  as  if  all 
previous  nodes  were  successful.  If  that  is  the  case,  then  there  would  be  no  need  to  Recall 
Standby/Retired  Reserve  because  the  previous  node  Recall  Selected/Ready  Reserve 
would  have  already  provided  all  required  Airmen.  Obviously  there  are  important 
dependencies  between  these  nodes  that  are  not  captured  in  the  current  PSM  models. 
Additionally,  the  percent  of  the  required  surge  that  gets  filled  can  affect  the  successful 
completion  of  nodes  that  require  the  use  of  those  reservists.  For  all  of  these  reasons, 
these  nodes  lent  themselves  to  being  evaluated  with  the  proposed  ESMs.  Nodes  were 
selected  to  show  how  the  quantity  of  Airmen  recalled  will  affect  the  overall  probability  of 
success.  The  nodes  selected  for  this  were  from  the  Flying  Training  Block,  Generate 
Training  Mission  (8.02),  Undergraduate  Flying  Training  (8.03),  Graduate  Flying 
Training  (8.04),  Mission  Flying  Training  (8.05),  and  Flying  Deployment  Training  (8.06). 
Next,  it  was  decided  that  showing  the  use  of  the  activation  node  would  be  beneficial. 
Therefore,  Acquire  Material?  (5.05)  and  the  node  it  activates  Obtain  Material  (5.06) 
were  included  in  the  model.  Lastly,  ESMs  can  take  into  consideration  environmental 
factors  that  might  have  an  impact  on  the  overall  probability  of  success;  this  allows  the 
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CRRA  practitioner  to  evaluate  the  importance  of  those  environmental  factors.  To 
provide  an  example  of  this  new  feature,  the  Force  Protection  Condition  (FPCON)  is 
included.  The  resulting  model  is  shown  in  Figure  4.7. 


Figure  4.7  Model  Verification  of  a  segment  of  ACS  ESM 
A  function  node  was  used  to  handle  the  relationships  between  surge  requirements  and 
reserves  recalled.  The  non-controlling  nodes  are  dependent  on  the  percent  of  the  surge 
requirement  filled.  If  the  surge  requirement  is  less  than  the  number  of  people  that  can  be 
obtained  through  both  forms  of  reserves,  the  percentage  is  100%  filled.  If  the  surge 
requirement  is  greater  than  the  number  of  people  that  can  be  supplied  from  the  reserves, 
the  percent  filled  is  calculated  and  used  to  control  the  probability  of  success  of  all  non¬ 
controlling  nodes. 

With  the  model  framework  established,  two  mock  SMEs  were  selected  to  score 
the  nodes.  It  should  be  emphasized  these  SMEs  were  not  actually  experts  in  the  areas 
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they  scored.  The  scores  presented  are  notional  and  in  no  way  should  be  considered 


indicative  of  actual  Air  Force  capability  or  deficiency.  A  scenario  was  presented  to  the 
SMEs  and  a  series  of  questions  were  asked  to  obtain  scores  for  the  model.  Their  answers 
were  then  input  to  a  computer  program  developed  in  MATLAB®  during  this  research  to 
generate  the  overall  probability  of  successfully  completing  all  of  the  non-controlling 
nodes  and  the  sensitivity  each  node  has  on  the  overall  probability  of  success.  The 
MATLAB®  code  is  presented  in  Appendix  B.  The  complete  lists  of  questions  asked  of 
the  SMEs  and  the  answers  they  provided  are  presented  in  Appendix  C.  A  sample  of  the 
questions  and  answers  is  presented  below. 

Table  4.2.  SME  questions  and  responses 


Identify  Surge  Requirements  (reported  in  numbers  of  peop 

le) 

Given  the  scenario,  what  is  the  lowest  likely  number  of  surge 
personnel  required? 

20 

Given  the  scenario,  what  is  the  most  likely  number  of  surge 
personnel  required? 

60 

Given  the  scenario,  what  is  the  largest  likely  number  of  surge 
personnel  required? 

80 

Recall  Ready  Reserve  (reported  in  numbers  of  people) 

Given  the  scenario,  what  is  the  lower  limit  on  the  expected 
number  of  Ready  Reserve  that  can  be  recalled? 

0 

Given  the  scenario,  what  is  the  expected  number  of  Ready 
Reserve  that  can  be  recalled? 

30 

Given  the  scenario,  what  is  the  upper  limit  on  the  expected 
number  of  Ready  Reserve  that  can  be  recalled? 

40 

Recall  Standby/Retired  Reserve  (reported  in  numbers  of  peo 

pie) 

Given  the  scenario,  what  is  the  lower  limit  on  the  expected 
number  of  Standby/Retired  Reserve  that  can  be  recalled? 

0 

Given  the  scenario,  what  is  the  expected  number  of 
Standby/Retired  Reserve  that  can  be  recalled? 

2 

Given  the  scenario,  what  is  the  upper  limit  on  the  expected 
number  of  Standby/Retired  Reserve  that  can  be  recalled? 

5 
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Based  on  the  input  from  the  SMEs,  triangular  probability  density  functions  (pdfs) 
were  established  and  10,000  samples  were  taken  from  each.  10,000  samples  were  taken 
to  consistently  provided  samples  across  the  entire  range  of  possible  inputs.  For  example, 
because  of  the  low  probability  of  obtaining  samples  of  20  or  80  in  the  Required  Surge 
Distribution  presented  in  Table  4.2,  less  than  10,000  runs  would  consistently  produce 
sample  sets  that  never  include  20  or  80.  The  number  of  runs  will  have  to  change  for 
future  applications  of  this  method  to  ensure  the  full  range  of  possible  inputs  is 
represented.  Histograms  of  the  values  of  each  of  the  samples  are  presented  in  Figures 
4.8,  4.9,  and  4.10. 
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Ready  Reserve 


Figure  4.9  Ready  Reserve  Distribution 


Figure  4.10  Retired  Reserve  Distribution 

The  sample  values  presented  above  were  used  to  find  the  percent  of  the  requirement  filled 
for  each  sample  group.  The  result  is  presented  in  Figure  4.11. 
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Figure  4.1 1  Percent  of  Surge  Filled 


Figure  4.1 1  shows  that  the  vast  majority  of  runs  (approximately  9,600  of  the 
10,000  runs)  produced  less  than  100%  of  the  required  surge.  This  is  true  even  though 
more  runs  equaled  100%  than  any  other  single  percentage.  This  can  be  seen  by  the  large 
bar  located  at  100  in  Figure  4.1 1.  The  reason  so  many  more  runs  resulted  in  100%  is  that 
for  all  situations  where  the  number  of  people  required  for  the  surge  was  less  than  the 
number  of  reservists  that  could  be  recalled  the  percentage  was  assigned  100%.  Without 
this  rule,  Figure  4.1 1  would  have  values  that  extend  beyond  the  100%  mark.  Being  able 
to  recall  more  than  what  is  required  is  not  important  to  the  effects  on  the  dependent  nodes 
because  it  was  assumed  only  the  number  of  reservists  required  would  actually  be  recalled. 
Therefore,  the  ability  to  recall  more  than  the  number  required  represents  100%  of  the 
required  number  recalled. 
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The  relationships  between  each  of  the  non-controlling  nodes  and  the  percent  of 


surge  filled  and  FPCON  were  found  by  asking  the  SMEs  to  score  each  non-controlling 
node  based  on  the  different  values  of  the  controlling  nodes.  The  SMEs  also  provided  the 
bounded  range  on  the  pdf  for  each  non-controlling  node.  The  bounded  range  presents  the 
SMEs  uncertainty  of  the  P(S)  they  provided.  A  sample  of  the  questions  asked  and  the 
SME  answers  are  presented  in  Table  4.3.  The  complete  questions  and  answers  are 
presented  in  Appendix  C. 

Table  4.3  SME  dependency  questions  and  responses 


Generate  Training  Mission 

What  is  the  probability  of  Successfully  completing  'Generate  Training 

Mission'  based  on  the  FPCON  and  %  surge  filled  provided? 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.96 

0.03 

ALPHA(l) 

0 

0.65 

DELTA(4) 

0 

0.7 

Undergraduate  Flying  Training 

What  is  the  prob 
Flying  Training' 

ability  of  Successfully  coi 
based  on  the  FPCON  anc 

tnpleting  ’Undergraduate 
l  %  surge  filled  provided? 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.98 

0.03 

ALPHA(l) 

0 

0.6 

DELTA(4) 

0 

0.5 

Mission  Flying  Training 

What  is  the  probability  of  Successfully  completing  ’Mission  Flying 

1  Training’  based  on  the  FPCON  and  % 

surge  filled  provided?  | 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.99 

0.03 

ALPHA(l) 

0 

0.7 

DELTA(4) 

0 

0.6 
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The  inputs  listed  in  the  table  4.3  were  used  to  approximate  linear  dependency 
relationships  between  all  of  the  non-controlling  and  controlling  nodes.  The  equations 
describing  each  of  those  relationships  are  presented  below. 

Generate  Training  Mission  =  0.6333  +  0.0167(FPCON)  +  0.31(Percent  Surge  Filled) 
Obtain  Training  Material  =  0.9333  -  0.0833(FPCON)  +  0.15(Percent  Surge  Filled) 
UFT  =  0.633  -  0.0333(FPCON)  +  0.38(Percent  Surge  Filled) 

GFT  =  0.633  -  0.0333  (FPCON)  +  0.38(Percent  Surge  Filled) 

MFT  =  0.733  -  0.0333  (FPCON)  +  0.29(Percent  Surge  Filled) 

FDT  =  0.7833  -  0.0333  (FPCON)  +  0.24(Percent  Surge  Filled) 

The  percent  of  the  surge  filled  values  computed  above  were  combined  with  the 
FPCON  levels  to  find  the  overall  probability  of  success  for  the  entire  model.  For  this 
example,  the  equation  to  find  the  overall  probability  of  success  is  the  product  of  all  non¬ 
controlling  nodes  with  Obtain  Training  Material  weighted  to  reflect  how  often  it  is 
activated.  The  resulting  distribution  reveals  the  most  likely  probability  of  success  of  the 
activities  in  this  scenario  is  approximately  20%  as  presented  in  Figure  4.12. 
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Overall  Probability  of  Success  Distribution 

i - 1 - 1 - 1 - 1 - 1 - r 


Probability  of  Success 

Figure  4.12  Overall  Probability  of  Success  Distribution 
Next,  a  sensitivity  analysis  was  conducted  to  determine  the  affect  each  node  has 
on  the  outcome  of  the  overall  probability.  The  sensitivity  analysis  used  was  the 
availability  method  explained  in  Section  4.3.  The  result  of  the  sensitivity  analysis  is 
shown  in  Figure  4.13.  The  node  that  has  the  most  affect  on  the  overall  probability  is  the 
Surge  Required  Node  followed  by  the  Ready  Reserve  Node.  For  this  example  the  node 
that  was  shown  to  most  affect  the  overall  probability  turned  out  to  be  a  node  outside  of 
the  control  of  the  Air  Force.  The  surge  required  is  largely  dictated  by  the  scenario.  This 
does  not  suggest  that  it  is  not  important  to  consider.  Understanding  what  the  leading 
drivers  of  the  system  are  even  when  they  are  outside  the  control  of  the  system  lends  itself 
to  solutions  that  don’t  rely  so  strongly  on  that  external  factor.  For  example,  if  an  analysis 
is  conducted  that  shows  the  weather  conditions  are  a  significant  factor  in  the  successful 
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completion  of  the  mission,  it  could  point  toward  developing  systems  that  can  more 


reliability  work  in  all  weather  situations. 


79 


5.  Conclusion 

5.1  Overview 

The  Air  Force  has  established  an  analysis  method  for  evaluating  the  capabilities  it 
is  expected  to  provide  to  the  Joint-warfighter.  It  does  this  by  combining  the  insights  of 
Subject  Matter  Experts  (SMEs)  with  a  framework  represented  in  Process  Sequence 
Models  (PSMs)  to  determine  which  capabilities  cause  the  Air  Force  the  greatest  degree  of 
risk.  Ultimately,  resource  allocation  decisions  are  made  based  on  this  evaluation.  Over 
the  last  six  years  the  Capability  Review  and  Risk  Assessment  (CRRA)  has  gone  through 
several  iterations.  Each  iteration  brought  with  it  changes  that  were  intended  to  uncover 
more  information  relevant  to  Air  Force  performance  and  increase  the  validity  and 
repeatability  of  the  process.  The  current  CRRA  method  has  come  a  long  way  since  its 
inception  but  remains  constrained  by  simplifying  assumptions  on  the  models  that  limit  its 
insight  into  the  problem  under  examination  while  creating  avenues  for  critics  to  question 
the  validity  of  its  output.  As  this  research  pointed  out,  the  most  limiting  assumptions 
made  are: 

•  Each  activity  in  a  PSM  can  have  only  two  states;  success  or  failure. 

•  The  degree  by  which  an  activity  succeeds  or  fails  has  no  impact  on  the 
overall  mission  completion. 

•  Each  activity  is  evaluated  assuming  all  other  activities  in  the  PSM  were 
successful,  thereby  assuming  independence  in  the  scores. 

Consequences  of  these  assumptions  are  that  the  effect  of  capability  quantity  or 
sufficiency  is  not  considered  and  there  is  no  representation  of  dependencies  or  how 
dependencies  can  affect  mission  completion. 
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The  method  of  Extended  Sequence  Modeling  has  been  presented  that  relaxes 
these  assumptions.  ESMs  allow  CRRA  practitioners  to  examine  the  dependent 
relationships  of  the  system  and  discover  how  changes  in  dependent  activities  can  ripple 
through  a  system  to  affect  the  overall  outcome.  Additionally,  Air  Force  planners  can  now 
explore  how  the  quantity  of  a  capability  or  how  well  it  is  accomplished  can  impact 
mission  success.  It  does  this  by  empowering  the  CRRA  practitioner  with  the  ability  to 
choose  which  dependencies  are  of  interest  to  examine  and  which  capabilities  or  activities 
are  better  modeled  using  a  graduated  scale  of  sufficiency  rather  than  probability  of 
success.  It  then  uses  input  from  SMEs  to  determine  the  strength  of  the  dependency  each 
of  the  dependent  nodes  have  with  their  controlling  nodes.  This  SME  input  combined 
with  the  techniques  of  reliability  analysis  and  linear  regression  establishes  linear 
relationships  between  the  dependent  and  controlling  nodes. 

One  of  the  strengths  of  this  new  method  is  it  can  be  added  to  the  existing  PSMs. 

If  the  CRRA  practitioner  wishes  to  keep  the  current  modeling  technique  intact  but  is  still 
curious  of  a  node’s  dependency  on  a  particular  controlling  variable,  the  probability 
density  function  of  that  one  node  can  be  established  using  the  techniques  of  ESMs  while 
leaving  the  rest  of  the  PSM  unchanged.  Additionally,  ESMs  can  be  used  to  stretch  across 
traditional  CONOPS  lines.  For  example,  the  outcome  of  Agile  Combat  Support 
Activities  can  be  linked  to  activities  within  operational  CONOPS  models.  Therefore, 
when  a  sensitivity  analysis  is  performed,  it  could  result  in  evidence  for  the  importance  of 
the  supporting  functions.  As  is  the  case  with  all  models,  this  new  model  also  makes 
simplifying  assumptions,  albeit  somewhat  less  restrictive  ones  than  the  current  model. 

The  most  significant  assumptions  are  that  linear  relationships  exist  between  dependent 
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and  controlling  nodes  and  that  all  nodes  not  specifically  shown  to  be  dependent  in  the 
model  are  independent. 

While  this  method  was  presented  to  show  the  minimum  amount  of  information 
required  from  the  SME,  it  was  also  shown  that  the  method  can  be  expanded.  By 
accepting  additional  information  from  SMEs  it  is  possible  to  arrive  at  a  more  accurate 
representation  of  the  dependency  relationships.  Also,  the  expanded  technique  can  avoid 
the  potential  for  group  think  that  can  exist  when  a  group  of  SMEs  are  forced  to  come  to  a 
consensus.  This  model  formulation  can  easily  fit  a  linear  relationship  to  large  numbers  of 
SMEs,  each  contributing  their  own  assessments  of  dependencies. 

The  Air  Force  is  a  large,  complicated  organization.  Modeling  its  performance 
entails  understanding  its  various  components,  how  they  interact  with  each  other,  to 
include  the  rules  they  follow  and  are  constrained  by,  and  how  these  factors  change 
through  time.  It  is  exactly  this  sort  of  system  that  allows  the  benefits  of  Enterprise 
Architecture  to  be  fully  appreciated.  To  get  the  full  benefits  from  architecture  there  are  a 
few  considerations  that  must  be  made.  The  first  is  the  architecture  should  be  able  to 
contain  all  information  that  is  relevant  to  the  processes  and  people  who  use  it.  This  does 
more  than  merely  save  people  the  hassle  of  jumping  from  one  program  to  another  to  view 
all  information  they  need.  By  having  all  information  in  one  repository  it  allows 
relationships  to  be  discovered  and  established  which  creates  additional  benefits,  some  of 
which  were  described  in  Chapter  4.  Additionally,  while  architecture  with  a  robust  meta 
model  to  handle  a  wide  variety  of  information  types  is  essential,  the  presentation  of  that 
information  needs  to  be  adjustable.  Many  different  people  use  architecture  for  different 
purposes.  The  user  needs  to  be  able  to  generate  views,  models,  or  other  types  of  reports 
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that  are  in  agreement  with  their  purposes.  In  order  to  contain  all  of  the  required 
information  it  is  likely  the  architecture  itself  would  have  to  be  maintained  at  a  classified 
level.  For  example,  for  an  Air  Force  Enterprise  Architecture  to  fully  support  the  CRRA, 
it  would  have  to  capture  the  scores  provided  by  SMEs.  These  scores  are  classified  and 
therefore  the  architecture  would  need  to  be  classified. 

The  Air  Force’s  Office  of  Warfighting  Integration  and  Chief  Information  Officer 
(SAF/XC)  has  developed  guidelines  that  allow  architectures  that  are  built  throughout  the 
Air  Force  to  be  “Fit  for  Federation”.  This  means  all  architectures,  whether  they  are  for 
systems,  organizations,  or  services  can  be  interfaced  with  higher  level  architectures. 
Through  these  defined  interfaces  a  user  can  start  at  a  top  level  architecture  and  query, 
search,  or  explore  down  into  all  of  the  architectures  that  are  federated  to  it.  The  power  of 
this  technique  can  be  appreciated  when  viewed  in  application  to  the  CRRA.  Chapter  4 
explored  using  an  SV-5  and  SV-7  to  support  the  CRRA.  Within  a  federated  architecture, 
the  architecture  could  be  queried  to  produce  an  SV-5  and  SV-7  of  all  systems  or  even 
organizations  that  provide  a  designated  capability  and  the  performance  of  that  system. 

With  respect  to  the  model  used  for  the  CRRA  analysis,  benefits  of  enterprise 
architecture  can  be  seen  in  two  overarching  ways.  The  first  is  assisting  the  construction 
and  scoring  of  the  model  to  be  evaluated.  The  Air  Force  architecture  can  contain 
information  that  can  be  reused  for  many  different  purposes  including  the  CRRA.  By 
pulling  information  from  the  architecture,  models  can  be  constructed  that  represent  the 
parts  of  the  Air  Force  to  be  studied  by  the  CRRA.  As  already  highlighted,  the  Draft 
DoDAF  2.0  contains  new  viewpoints  that  can  be  used  to  assist  with  the  CRRA.  The 
second  benefit  of  enterprise  architecture  is  its  ability  to  capture  the  model  used  for  the 
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evaluation  and  store  it  in  the  architecture  to  be  used  for  future  CRRA  analysis  or  for  other 
unforeseen  purposes. 

5.2  Recommendations  for  Future  Research 

The  following  items  are  suggested  for  further  study. 

1)  Once  DoDAF  version  2.0  becomes  final,  a  thorough  analysis  should  be 
conducted  to  understand  how  its  expanded  viewpoints  and  data  model  can 
assist  the  CRRA.  Likewise,  recommendations  to  add  ESM  to  future 
DoDAF  versions  could  begin. 

2)  Investigate  a  way  to  accurately  assess  the  PAARs  and  implement  a 
process  to  further  research  areas  where  additional  risk  may  be  accepted. 

3)  The  CRRA  is  performed  within  each  CONOP  with  limited  connection 
between  the  CONOPs.  Although  the  effects  of  a  mission  area  in  one 
CONOP  on  another  CONOP  is  starting  to  be  examined,  there  still  exists 
disconnects.  Research  can  be  performed  to  determine  how  best  to  align 
the  various  missions  under  scrutiny  within  the  CRRA.  A  study  should  be 
undertaken  on  how  the  ESM,  and  other  techniques,  can  be  used  to 
combine  the  effects  of  multiple  CONOPs.  This  concept  could  support  the 
mathematical  analysis  of  an  integrated  CRRA  across  CONOPs. 

4)  One  concern  from  a  CRRA  practitioner  was  the  lack  of  a  technique  to 
determine  which  shortfalls  across  the  multiple  CONOPS  should  be  the 
highest  priority.  AF/A5X-C  has  started  to  address  this  issue.  It  would  be 
beneficial  to  research  a  method  for  assessing  shortfall  priority  across 
various  CONOPs. 
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5)  A  study  should  examine  the  effects  to  opening  up  the  CRRA  to  many 
more  scoring  SMEs  for  the  purpose  of  getting  more  points  of  view. 

6)  This  ESM  approach  uses  linear  relationships.  The  modeling  of  ESM 
dependencies  using  non-linear  relationships  could  be  examined. 

7)  Currently  the  final  PSM  result  is  a  probability  of  success  score  that  is 
related  to  a  consequence  by  the  combatant  commands.  It  is  a  natural 
extension  of  this  research  to  show  how  other  variables  other  than 
probability  of  success  can  be  linked  to  consequences.  Through  this, 
sufficiency  of  a  capability  can  be  linked  to  consequence  similar  to  this 
research’s  aim  to  link  sufficiency  of  a  capability  to  probability  of  success. 

8)  Future  work  could  study  the  statistical  properties  of  the  SME  input  data 
when  finding  the  dependency  relationships  for  the  ESM  method  to 
determine  if  such  information  can  be  of  use  in  the  analysis.  For  example, 
the  standard  deviation  of  their  scores  might  reveal  information  about  the 
nature  of  dependent  relationships  or  the  uncertainty  associated  with  those 
relationships. 

9)  Implementation  of  the  ESM  method  should  be  demonstrated  for 
distributions  other  than  triangular.  The  behavior  of  normal  distributions 
undergoing  linear  transformations  is  well  understood  (they  remain 
normal),  but  other  distributions  may  be  necessary  for  cases  where  hard 
constraints  bound  possible  values. 
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Appendix  A.  Icon  Taxonomy 


- -> 

Dependency 

Relationship 

The  pdf  for  the  node  at  the  head  of  the  arrow  is 
linearly  dependent  on  the  variable  of  the  node 
at  the  tail  of  the  arrow. 

- > 

Data 

Relationship 

The  value  of  the  variable  for  the  node  at  the 
tail  of  the  arrow  is  input  to  the  node  at  the 
heard  of  the  arrow  -  can  only  feed  to  a 
function  node. 

Series  Node 
Connector 

Connects  non-controlling  nodes  in  series. 

=► 

Activation 

Relationship 

Tail  is  always  linked  to  activation  node,  head 
is  always  linked  to  a  non-controlling  node.  It 
shows  which  non-controlling  Node  is  activated 
by  the  Activation  Node. 

-c 

Redundant 

Node 

Connector 

Displays  two  or  more  nodes  that  are  redundant 

Controlling 

Node 

Can  represent  any  activity  or  environmental 
factor  that  has  a  variable  that  will  affect 
another  node.  Can  be  linked  to  a  non¬ 
controlling  or  other  controlling  nodes  by  a 
Dependency  Relationship  or  to  a  function  by  a 
data  relationship.  Variable  can  be  P(s)  or  any 
other  metric.  They  only  affect  overall  mission 
P(s)  indirectly  through  non-controlling  nodes. 

Controlling 

Node 

/  Activation  \ 

N.  Node  ^ 

Activation 

Node 

Contains  a  percentage  that  controls  the 
weighted  strength  of  non-controlling  nodes. 
Percentage  usually  dependent  on  the  scenario. 

Function 

Node 

Function 

Node 

Has  a  dependent  variable  that  is  determined  by 
the  input  variable(s)  and  the  function  logic. 

*  Must  have  one  or  more  inputs 

*  Output  can  be  Data  Relationship  or 
Dependency  Relationship 

Non 

Controlling 

Node 

Non- 

Controlling 

Node 

Variable  must  be  P(s).  Directly  impacts  the 

P(s)  of  the  overall  mission. 

Container  Node 

Container 

Node 

Groups  similar  nodes. 

Mission 

1 

Mission 

2 
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Appendix  B.  MATLAB  Code 


%Written  by  Peter  Mastro  28  January  2009 

%Calculates  overall  probability  of  success  for  the  sample  model 
presented 

%to  our  thesis  advisors  on  27  January  2009,  for  the  purpose  of 

validating 

%the  modeling  method 

clear  %clears  all  variables 

runs=10000;  %sets  the  number  of  samples  that  will  be  taken  of  each 
variable 

disp('This  program  will  calculate  the  overall  probability  of  success  of 
the  mission  presented  to  the  Advisors  on  Jan  27’) 

%The  following  block  receives  the  bounds  for  the  expected  surge 
requirement 

%,  builds  the  triangular  distribution  and  picks  random  values  from  it 
lowest_req=input (' Enter  the  lowest  expected  surge  requirement:  '); 

most_req=input (' Enter  the  most  likely  surge  requirement:  ' ); 

highest_req=input (' Enter  the  highest  likely  surge  requirement:  '); 

a  =  lowest_req; 
b  =  highest_req; 
c  =  most_req; 

u  =  rand (runs , 1 ) ;  %selects  random  numbers 

lo  =  (u<= ( (c-a) / (b-a) ) ) ;  %determines  all  random  numbers  that  are  less 
than  the  mode 

req(lo)  =  a  +  sqrt (u ( lo ) . * ( (b-a ) * ( c-a ) ) ) ;  %uses  the  inversion  method 
with  the  random  numbers  to  generate  triangular  dist. 
req(-lo)  =  b  -  sqrt (( 1-u ( ~lo) ).* (b-a) * (b-c) ) ; 

req=round (req) ; %rounds  to  the  nearest  whole  number  since  we  are  dealing 
with  people 

hist (req, ( (b-a) +1 ) ) %displays  histogram  that  has  the  same  number  of  bins 

as  there  are  number  of  different  possible  amounts 

xlabel (' Number  of  People  Required  for  the  Surge FontSize f , 12 ) 

ylabel (' Frequency  number  was  selected FontSize ', 12 ) 

t it le (' Required  Surge  Distribution FontSize ', 16 ) 

disp  (  ’  '  ) 

%The  following  block  receives  the  bounds  for  the  expected  ready  reserve 
%,  builds  the  triangular  distribution  and  picks  random  values  from  it 
lowest_redres=input (' Enter  the  lower  limit  on  expected  Ready  Reserve  to 
be  recalled :  ’ ) ; 

most_redres=input (' Enter  the  most  likely  number  of  Ready  Reserve  to  be 
recalled :  ’ ) ; 

highest_redres=input (' Enter  the  upper  limit  on  expected  Ready  Reserve 
to  be  recalled:  '); 

a  =  lowest_redres ; 
b  =  highest_redres ; 
c  =  most_redres; 
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u  =  rand (runs , 1 ); %selects  random  numbers 

lo  =  (u< ( ( c-a ) / (b-a ) ) ) ; %determines  all  random  numbers  that  are  less 
than  the  mode 

redres(lo)  =  a  +  sqrt(uflo) .*( (b-a)*(c-a) ) ) ;%uses  the  inversion  method 
with  the  random  numbers  to  generate  triangular  dist. 
redres ( ~lo )  =  b  -  sqrt ( ( 1-u (~lo) ) . * (b-a) * (b-c) ) ; 

redres=round (redres ) ; %rounds  to  the  nearest  whole  number  since  we  are 
dealing  with  people 

hist (redres,  ( (b-a ) +1 ) ) %displays  histogram  that  has  the  same  number  of 
bins  as  there  are  number  of  different  possible  amounts 
xlabel (' Number  of  People  received  from  the  Ready 
Reserve', ' FontSize ' , 12 ) 

ylabel (' Frequency  number  was  selected FontSize ', 12 ) 
title ( 1  Ready  Reserve  5 ,  5  FontSize ' , 16 ) 
disp('  ') 

%The  following  block  receives  the  bounds  for  the  expected  retired 
reserve 

%,  builds  the  triangular  distribution  and  picks  random  values  from  it 
lowest_retres=input (' Enter  the  lower  limit  on  expected  Retired  Reserve 
to  be  recalled:  '); 

most_retres=input (' Enter  the  most  likely  number  of  Retired  Reserve  to 
be  recalled :  '  )  ; 

highest_retres=input (' Enter  the  upper  limit  on  expected  Retired  Reserve 
to  be  recalled:  '); 

a  =  lowest_retres ; 
b  =  highest_retres ; 
c  =  most_retres; 

u  =  rand (runs , 1 ); %selects  random  numbers 

lo  =  (u< ( (c-a) / (b-a) ) ) ;%determines  all  random  numbers  that  are  less 
than  the  mode 

retres(lo)  =  a  +  sqrt (u ( lo ). * ( (b-a )*( c-a ))); %uses  the  inversion  method 
with  the  random  numbers  to  generate  triangular  dist. 
retres(-lo)  '=  b  -  sqrt (( 1-u ( ~lo) ).* (b-a) * (b-c) ) ; 

retres=round (retres ) ; %rounds  to  the  nearest  whole  number  since  we  are 
dealing  with  people 

hist (retres, ( (b-a ) +1 ) ) %displays  histogram  that  has  the  same  number  of 
bins  as  there  are  number  of  different  possible  amounts 

xlabel (' Number  of  People  received  from  Retired  Reserve FontSize ', 12 ) 
ylabel (' Frequency  number  was  selected FontSize ', 12 ) 
title ( ' Retired  Reserve ' , ' FontSize ' , 16 ) 
disp  (  '  '  ) 

result=input (' press  return  when  ready'); 

%The  following  block  takes  the  random  numbers  previously  generated  and 
%calculates  the  percent  of  the  required  surge  that  is  met  by  the  ready 
and 

%retired  reserves,  then  displays  a  histogram  of  the  information, 
for  n=l : runs 

if  (req (n) <= (redres (n) iretres (n) ) ) 

percent_f ill (n) =  1 ;  %if  there  are  more  people  than  required  it 
reports  100% 
else 
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percent_f ill (n) =  ( (redres (n) iretres (n) ) /req(n) )  ;%  if  there  are  less 
people  than  required  this  finds  the  percent  filled 
end 
end 

hist (percent_f ill, 100 ) %  displays  histogram 

xlabel (' Percent  of  the  Surge  Filled ' ,  ' FontSize ', 12 )  %label  for 
histogram 

ylabel (' Frequency  percentage  was  computed FontSize f , 12 )  %label  for 
histogram 

title ( 1  Percent  of  Surge  Filled ' ,  ' FontSize ' , 16 )  %title  for  histogram 
disp  (  ’  ’) 

%This  receives  the  FPCON  information  then  develops  an  array  with  the 
same  number  of 

%each  level  as  inputed  by  the  percentage  inputted  by  the  user 
FPCON_Alpha=input ( f What  is  the  probability  the  base  will  be  at  FPCON 
Alpha?  ' ) ; 

FPCON_Bravo=input ( ' What  is  the  probability  the  base  will  be  at  FPCON 
Bravo?  ’ ) ; 

FPCON_Char lie=input ( ' What  is  the  probability  the  base  will  be  at  FPCON 
Charlie?  ’ ) ; 

FPCON_Delta=input ( f What  is  the  probability  the  base  will  be  at  FPCON 
Delta?  ’ ) ; 

c=l ; 

for  n=l : round (runs *FPCON_Alpha ) 

FPCON (c)=l;  %fills  an  array  to  the  appropriate  percentage  with  1  which 
stands  for  Alpha 
c=c+l ; 
end 

for  n=l : round (runs *FPCON_Bravo ) 
c=c  +  l ; 

FPCON ( c ) =2 ; %f ills  an  array  to  the  appropriate  percentage  with  2 
which  stands  for  Bravo 
end 

for  n=l : round ( runs *FPCON_Char lie ) 
c=c+l ; 

FPCON ( c ) =3 ; %f ills  an  array  to  the  appropriate  percentage  with  3 
which  stands  for  Charle 
end 

for  n=l : round (runs*FPCON_Delta) 
c=c+l ; 

FPCON ( c ) =4 ; %f ills  an  array  to  the  appropriate  percentage  with  4 
which  stands  for  Delta 
end 

hist (FPCON, 100) %  shows  the  histogram  of  the  FPCONS 
xlabel ( ! FPCON  levels Font Size ’, 12 ) %label  for  histogram 
ylabel (' Frequency  level  was  selected ' ,  ' FontSize ', 12 ) %label  for 
histogram 

title ('FPCON  level  distribution FontSize ', 16 ) 
disp ( ’  ') 

%gets  the  probability  obtain  materials  will  be  calculated  in  the  final 
%calculat ion 
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Acquire_mat=input ( ' What  is  the  probability  additional  training 
materials  will  be  required?  ’ ) ; 

%The  next  block  gets  the  probability  of  success  for  the  non-controlling 
%nodes  from  the  user 

if  ( lowest_req< (highest_redres+highest_retres ) ) 

greatest_percent_f illed=l ; 

else 

greatest_percent_f illed= ( (highest_redres+highest_retres ) /lowest_req) ; 
end 

if  (highest_req< ( lowest_redres+lowest_retres ) ) 

lowest_percent_f illed=l ; 

else 

lowest_percent_f illed= ( ( lowest_redres+lowest_retres ) /highest_req) ; 

end 

disp  (  '  '  ) 

disp('For  Generate  Training  Mission:  ') 

disp (’If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( great est_percent_f i lied* 100 ) 

Gen_Miss ( 1 ) =input ( ' what  is  the  probability  of  success?  '); 
disp  (  '  '  ) 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( lowest_percent_f i lied* 100 ) 

Gen_Miss ( 2 ) =input ( ' what  is  the  probability  of  success?  '); 
disp  (  '  ') 

disp ('If  the  FPCON  is  Delta  and  the  %  surge  filled  is') 
disp ( lowest_percent_f i lied* 100 ) 

Gen_Miss ( 3 ) =input ( ' what  is  the  probability  of  success?  '); 
GenMissRange=input ( ' What  is  the  of  the  probability  range?  '); 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2- 

disp  (  '  '  ) 

disp('For  Obtain  Training  Material:  ') 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( great est_percent_f i lied* 100 ) 

ob_Mat ( 1 ) =input ( ' what  is  the  probability  of  success?  '); 
disp  (  '  '  ) 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( lowest_percent_f i lied* 100 ) 

ob_Mat ( 2 ) =input ( ' what  is  the  probability  of  success?  '); 
disp  (  '  ') 

disp ('If  the  FPCON  is  Delta  and  the  %  surge  filled  is') 
disp ( lowest_percent_f i lied* 100 ) 

ob_Mat ( 3 ) =input ( ' what  is  the  probability  of  success?  '); 
obMatRange=input ( ' What  is  the  of  the  probability  range?  '); 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2- 

disp  (  '  '  ) 

disp('For  Undergraduate  Flying  Training:  ') 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 

disp ( great est_percent_f i lied* 100 ) 

UFT ( 1 ) =input ( ' what  is  the  probability  of  success?  '); 
disp  (  '  '  ) 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( lowest_percent_f i lied* 100 ) 
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UFT ( 2 ) =input ( ' what  is  the  probability  of  success?  '); 
disp ( ’  ') 

disp  ('If  the  FPCON  is  Delta  and  the  %  surge  filled  is') 
disp ( lowest_percent_f illed*100 ) 

UFT ( 3 ) =input ( ' what  is  the  probability  of  success?  '); 
UFTRange=input  (  ' What  is  the  "  +  of  the  probability  range? 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2' 

disp  (  '  '  ) 

disp('For  Graduate  Flying  Training:  ') 

disp (’If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( great est_percent_f illed*100 ) 

GFT ( 1 ) =input ( ' what  is  the  probability  of  success?  '); 


disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( lowest_percent_f illed*100 ) 

GFT ( 2 ) =input ( ' what  is  the  probability  of  success?  '); 

disp ('If  the  FPCON  is  Delta  and  the  %  surge  filled  is') 
disp ( lowest_percent_f illed*100 ) 

GFT ( 3 ) =input ( ' what  is  the  probability  of  success?  '); 
GFTRange=input ( ' What  is  the  of  the  probability  range? 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2' 

disp  (  '  '  ) 

disp('For  Mission  Flying  Training:  ') 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is'  ) 
disp ( great est_percent_f illed*100 ) 

MFT ( 1 ) =input ( ' what  is  the  probability  of  success?  '); 
disp  (  '  '  ) 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is'  ) 
disp ( lowest_percent_f illed*100 ) 

MFT ( 2 ) =input ( ' what  is  the  probability  of  success?  '); 
disp ( '  ') 

disp ('If  the  FPCON  is  Delta  and  the  %  surge  filled  is'  ) 
disp ( lowest_percent_f illed*100 ) 

MFT ( 3 ) =input ( ' what  is  the  probability  of  success?  '); 
MFTRange=input ( ' What  is  the  of  the  probability  range? 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2- 

disp  (  '  '  ) 

disp('For  Flying  Deployment  Training:  ') 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( great est_percent_f illed*100 ) 

FDT ( 1 ) =input ( ' what  is  the  probability  of  success?  '); 
disp  (  '  '  ) 

disp ('If  the  FPCON  is  Alpha  and  the  %  surge  filled  is') 
disp ( lowest_percent_f illed*100 ) 

FDT ( 2 ) =input ( ' what  is  the  probability  of  success?  '); 
disp ( '  ') 

disp ('If  the  FPCON  is  Delta  and  the  %  surge  filled  is') 
disp ( lowest_percent_f illed*100 ) 

FDT ( 3 ) =input ( ' what  is  the  probability  of  success?  '); 
FDTRange=input ( ' What  is  the  of  the  probability  range? 


%The  next  block  finds  the  parameters  of  the  dependency  equation  for 
each  of 
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%the  non-controlling  nodes 

x= [1,  1,  gr eat e s t_per cent_f i lied; 1,1, 1 owe st_percent_fi lied; 1,4, lowest_per 
cent_f illed] ; %builds  the  input  variable  matrix 

y=Gen_Miss ’ ; 

GenMissB= ( (x 1 *x) A-l) * (x 1 *y ) ; 
y=ob_Mat ’ ; 

ob_MatB=  (  (x  1  *x)  A-l)  *  (x ' *y )  ; 
y=UFT ' ; 

UFTB= (  (x*  *x) A-l) * (x* *y)  ; 
y=GFT ’ ; 

GFTB= ( (x* *x) A-l) * (x* *y) ; 
y=MFT ' ; 

MFTB= ( (x* *x) A-l) * (x* *y) ; 
y=FDT ' ; 

FDTB= ( (x* *x) A— 1) * (x* *y) ; 


%This  finds  the  mode  of  the  pdf  for  each  of  the  runs  for  each  of 
%the  non-controlling  nodes 
for  n=l : runs 

GenMissProbmode (n) =GenMissB ( 1 ) +GenMissB ( 2 ) *FPCON (n) +GenMissB ( 3 ) ^percent 

_fill (n) ; 

end 

for  n=l : runs 

obMatProbmode (n) =ob_MatB ( 1 ) +ob_MatB ( 2 ) *FPCON (n) +ob_MatB ( 3 ) *percent_f ill 

(n)  ; 

end 

for  n=l : runs 

UFTProbmode ( n ) =UFTB ( 1 ) +UFTB (2) *FPCON(n) +UFTB (3) *per cent_f ill (n) ; 
end 

for  n=l : runs 

GFTProbmode ( n ) =GFTB ( 1 ) +GFTB ( 2 ) *FPCON(n) +GFTB ( 3 ) *per cent_f ill (n) ; 
end 

for  n=l : runs 

MFTProbmode (n)=MFTB(l) +MFTB ( 2 ) *FPCON(n) +MFTB ( 3 ) *per cent_f ill (n) ; 
end 

for  n=l : runs 

FDTProbmode (n) =FDTB (1) +FDTB (2) *FPCON(n) +FDTB (3) *per cent_f ill (n) ; 
end 
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%The  next  block  generates  triangular  distributions  based  on  the  modes 
generated  above  then  picks  a  random  number  from  the 
%distr ibut ion 

a  =  GenMissProbmode-GenMissRange ; 
b  =  GenMissProbmode+GenMissRange ; 
c  =  GenMissProbmode ; 

u  =  r and ( runs ^ 1) ;  %selects  random  number 
for  n=l : runs 

if  (u (n) < ( (c(n)-a(n) ) / (b(n)-a(n) ) ) ) ; 

GenMissProb (n) =  a(n)  +  sqrt (u (n) * ( (b (n) -a (n) ) * (c (n) -a (n) ) ) ) ; 
if  GenMissProb (n) >1 

GenMissProb (n) =1 ;  %if  because  of  the  Range  the  probability  is 
greater  than  1  it  is  assigned  1 
end 

if  GenMissProb (n) <0 

GenMissProb (n) =0 ;  %if  because  of  the  Range  the  probability  is  less 
than  0  it  is  assigned  0 
end 
else 

GenMissProb (n)  =  b(n)  -  sqrt ( ( 1-u (n) ) * (b (n) -a (n) ) * (b (n) -c (n) ) ) ; 
if  GenMissProb (n) >1 

GenMissProb (n) =1 ; %if  because  of  the  Range  the  probability  is 
greater  than  1  it  is  assigned  1 
end 

if  GenMissProb (n) <0 

GenMissProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less 
than  0  it  is  assigned  0 
end 
end 
end 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2- 
a  =  obMatProbmode-obMatRange ; 
b  =  obMatProbmode+obMatRange ; 
c  =  obMatProbmode ; 

u  =  r and ( runs / 1 ) ; % s elect s  random  numbers 
for  n=l : runs 

if  (u(n)<( (c(n)-a(n) )/ (b(n)-a(n)))); 

obMatProb (n) =  a(n)  +  sqrt (u(n) * ( (b(n)-a(n) ) * (c(n)-a(n) ) ) ) ; 
if  obMatProb (n) >1 

obMatProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  obMatProb (n) <0 

obMatProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less 
than  0  it  is  assigned  0 
end 
else 

obMatProb(n)  =  b(n)  -  sqrt ( ( 1-u (n) ) * (b (n) -a (n) ) * (b (n) -c (n) ) ) ; 
if  obMatProb (n) >1 

obMatProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  obMatProb (n) <0 

obMatProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less 
than  0  it  is  assigned  0 
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end 

end 

end 

'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O 

a  =  UFTProbmode-UFTRange ; 
b  =  UFTProbmode+UFTRange ; 
c  =  UFTProbmode; 

u  =  rand (runs ,  1 ); %selects  random  numbers 
for  n=l : runs 

if  (u(n)<( (c(n)-a(n) )/ (b(n)-a(n)))); 

UFTProb(n)=  a(n)  +  sqrt (u (n) * ( (b (n) -a (n)  ) * (c (n) -a (n)  )  )  )  ; 
if  UFTProb ( n ) >1 

UFTProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  UFTProb ( n ) <0 

UFTProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less  than 
0  it  is  assigned  0 
end 
else 

UFTProb(n)  =  b(n)  -  sqrt ( ( 1-u (n) ) * (b (n) -a (n) ) * (b (n) -c (n) ) ) ; 
if  UFTProb ( n ) >1 

UFTProb (n) =  1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  UFTProb ( n ) <0 

UFTProb (n) =0 ; %if  because  of  the  Range  teh  probability  is  less  than 
0  it  is  assigned  0 
end 
end 
end 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2' 

a  =  GFTProbmode-GFTRange ; 
b  =  GFTProbmode+GFTRange ; 
c  =  GFTProbmode; 

u  =  rand (runs ,  1 ); %selects  random  numbers 
for  n=l : runs 

if  (u(n)<( (c(n)-a(n) )/ (b(n)-a(n) ) ) )  ; 

GFTProb(n)=  a(n)  +  sqrt (u(n) * ( (b(n)-a(n) ) * (c(n)-a(n) ) ) ) ; 
if  GFTProb ( n ) >1 

GFTProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  GFTProb ( n ) <0 

GFTProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less  than 
0  it  is  assigned  0 
end 
else 

GFTProb(n)  =  b(n)  -  sqrt ( ( 1-u (n) ) * (b (n) -a (n) ) * (b (n) -c (n) ) ) ; 
if  GFTProb ( n ) >1 

GFTProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  GFTProb ( n ) <0 
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GFTProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less  than 
0  it  is  assigned  0 
end 
end 
end 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2- 

a  =  MFTProbmode-MFTRange  ; 
b  =  MFTProbmode+MFTRange ; 
c  =  MFTProbmode; 

u  =  r and ( runs / 1 ) ; % s elect s  random  numbers 
for  n=l : runs 

if  (u(n)<( (c(n)-a(n) )/ (b(n)-a(n)))); 

MFTProb(n)=  a(n)  +  sqrt (u(n) * ( (b(n)-a(n) ) * (c(n)-a(n) ) ) ) ; 
if  MFTProb ( n ) >1 

MFTProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  MFTProb ( n ) <0 

MFTProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less  than 
0  it  is  assigned  0 
end 
else 

MFTProb(n)  =  b(n)  -  sqrt ( ( 1-u (n) ) * (b (n) -a (n) ) * (b (n) -c (n) ) ) ; 
if  MFTProb ( n ) >1 

MFTProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  MFTProb ( n ) <0 

MFTProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less  than 
0  it  is  assigned  0 
end 
end 
end 

'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o 

a  =  FDTProbmode-FDTRange ; 
b  =  FDTProbmode+FDTRange ; 
c  =  FDTProbmode; 

u  =  rand (runs , 1 ); %selects  random  numbers 
for  n=l : runs 

if  (u(n)<( (c(n)-a(n) )/ (b(n)-a(n)))); 

FDTProb(n)=  a(n)  +  sqrt (u (n) * ( (b (n) -a (n) ) * (c (n) -a (n) ) ) ) ; 
if  FDTProb ( n ) >1 

FDTProb (n) =  1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 

if  FDTProb ( n ) <0 

FDTProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less  than 
0  it  is  assigned  0 
end 
else 

FDTProb(n)  =  b(n)  -  sqrt ( ( 1-u (n) ) * (b (n) -a (n) ) * (b (n) -c (n) ) ) ; 
if  FDTProb ( n ) >1 

FDTProb (n) =1 ; %if  because  of  the  Range  the  probability  is  greater 
than  1  it  is  assigned  1 
end 
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if  FDTProb ( n ) <0 

FDTProb (n) =0 ; %if  because  of  the  Range  the  probability  is  less  than 
0  it  is  assigned  0 
end 
end 
end 

%This  next  block  takes  all  of  the  probabilities  found  in  the  last 
section 

%and  multiplies  them  to  find  the  overall  probability 
for  n=l : runs 

totalprob (n) =GenMissProb (n) * ( (obMatProb (n) *Acquire_mat )  +  ( 1- 
Acquire_mat ) ) *UFTProb(n) *GFTProb(n) *MFTProb(n) *FDTProb ( n ) ; 
end 

hist  (totalprob, 100 ) %displays  the  final  probability 

xlabel ( f Probability  of  Success FontSize f , 12 ) %label  for  histogram 
ylabel (' Frequency  probability  was  computed FontSize 12 ) %label  for 
histogram 

t it le (' Overall  Probability  of  Success 
Distribution FontSize f , 16 ) %title  for  histogram 
%Written  by  Peter  Mastro  30  January  2009 

%This  program  finds  the  sensitivity  of  the  overall  probability  to 
changes 

%in  each  of  the  nodes 

SurgReq=lowest_req;  %sets  the  surge  required  to  the  lowest  value  it  can 
take  on 

ReadyRes=most_redres ; 

Ret iredRes=most_retres ; 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled=l; 

else 

f illed= (ReadyRes+Ret iredRes ) / SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) *f illed; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB  (1) +UFTB (2) *For cePr ot +UFTB ( 3 ) *f illed; 

GFTSens=GFTB (1) +GFTB (2) *For cePr ot +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For cePr ot +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For ceProt +FDTB ( 3 ) *f illed; 
SurgeLowtotal=GenMissionSens* ( ( Acquire_mat ^ObtainMatSens ) + (1- 
Acquire_mat ) ) ^UFTSens^GFTSens^MFTSens^FDTSens ;  %Finds  the  overall 
probability  based  on  the  lowest  surge  required 

disp('The  overall  probability  of  success  when  the  Surge  is  at  its 

lowest  is ’ ) 

disp ( SurgeLowtotal ) 

'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o 

SurgReq=highest_req; %sets  surge  required  to  the  most  it  can  be 
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ReadyRes=most_redres ; 

Ret iredRes=most_retres ; 

%calculates  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled*!; 

else 

f illed= (ReadyRes+Ret iredRes ) /SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) *f illed; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB (1) +UFTB (2) *For cePr ot +UFTB ( 3 ) *f illed; 

GFTSens=GFTB (1) +GFTB (2) *For cePr ot +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For ceProt +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For ceProt +FDTB ( 3 ) *f illed; 
SurgeHightotal=GenMissionSens* ( ( Acquire_mat *ObtainMatSens ) + (1- 
Acquire_mat ) ) *UFTSens*GFTSens*MFTSens*FDTSens ;  %finds  overall 
probability  based  on  the  highest  surge  value 

disp ('The  overall  probability  of  success  when  the  Surge  is  at  its 

highest  is ' ) 

disp ( SurgeHightotal ) 

SurgeSensdif f =abs ( SurgeHightotal-SurgeLowtotal ) ;%computes  the 
difference  between  the  two  extremes  of  the  surge 
disp ('The  probability  difference  is') 
disp ( SurgeSensdif f ) 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2- 


SurgReq=most_req; 

ReadyRes=lowest_redres ;  %sets  the  Ready  Reserve  to  the  lowest  value  it 
can  attain 

Ret iredRes=most_retres ; 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled*!; 

else 

f illed= (ReadyRes+Ret iredRes ) /SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) *f illed; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB  (1) +UFTB (2) *For ceProt +UFTB ( 3 ) *f illed; 

GFTSens=GFTB (1) +GFTB (2) *For ceProt +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For ceProt +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For ceProt +FDTB ( 3 ) *f illed; 
ReadyLowtotal=GenMissionSens * ( ( Acquire_mat *ObtainMatSens ) + (1- 
Acquire_mat ) ) *UFTSens*GFTSens*MFTSens*FDTSens ;  %calculates  the  overall 
probability  based  on  the  low  Ready  Reserve  number 
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disp ('The  overall  probability  of  success  when  the  Ready  Reserve  is  at 
its  lowest  is ’ ) 
disp (ReadyLowtotal ) 


SurgReq=most_req; 

ReadyRes=highest_redres ; 

Ret iredRes=most_retres ; 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled-! ; 

else 

f illed= (ReadyRes+Ret iredRes ) / SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) ^filled; 
ObtainMatSens=ob_MatB  ( 1 )  +ob_MatB  ( 2  )  *ForceProt+ob_MatB  ( 3  )  *  filled; 
UFTSens=UFTB  (1)  +UFTB  (2)  *For  cePr  ot  +UFTB  (  3  )  *f  Hied; 

GFTSens=GFTB (1) +GFTB (2) *For ceProt +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1)  +MFTB  (2)  *For  ceProt  +MFTB  (  3  )  *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For ceProt +FDTB ( 3 ) *f illed; 
ReadyHightotal=GenMissionSens* ( ( Acquire_mat *ObtainMatSens ) + (1- 
Acquire_mat ) ) *UFTSens*GFTSens*MFTSens*FDTSens ; % calculates  the  overall 
probability  based  on  the  high  Ready  Reserve  number 

disp ('The  overall  probability  of  success  when  the  Ready  Reserve  is  at 
its  highest  is ' ) 
disp (ReadyHightotal ) 

ReadySensdif f =abs (ReadyHightotal-ReadyLowtotal ) ;%computes  the 
difference  between  the  two  extremes  of  the  Ready  Reserve 
disp (’The  probability  difference  is') 
disp (ReadySensdif f ) 

2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2'2- 


SurgReq=most_req; 

ReadyRes=most_redres ; 

Ret iredRes=lowest_retres ;  %Sets  the  Retired  Reserve  to  the  lowest  it 
can  attain 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled=l; 

else 

f illed= (ReadyRes+Ret iredRes ) / SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) * filled; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
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UFTSens=UFTB ( 1 ) +UFTB (2) *For cePr ot +UFTB ( 3 ) ^filled; 

GFTSens=GFTB ( 1 ) +GFTB (2) *For cePr ot +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For cePr ot +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For cePr ot +FDTB ( 3 ) *f illed; 
RetiredLowtotal=GenMissionSens* ( ( Acquire_mat *ObtainMatSens ) + (1- 
Acquire_mat) ) *UFTSens*GFTSens*MFTSens*FDTSens ; 

disp ('The  overall  probability  of  success  when  the  Retired  Reserve  is  at 

its  lowest  is  1 ) 

disp (Ret iredLowtotal ) 

'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o 

SurgReq=most_req; 

ReadyRes=most_redres ; 

Ret iredRes=highest_retres ; %set s  the  Retired  Reserve  to  the  highest  it 
can  attain 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled=l; 

else 

f illed= (ReadyRes+Ret iredRes ) / SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) * filled; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB ( 1 ) +UFTB (2) *For cePr ot +UFTB ( 3 ) *f illed; 

GFTSens=GFTB (1) +GFTB (2) *For cePr ot +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For cePr ot +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For cePr ot +FDTB ( 3 ) *f illed; 

RetiredHightotal=GenMissionSens^ ( ( Acquire_mat ^ObtainMatSens ) + (1- 
Acquire_mat) ) *UFTSens*GFTSens*MFTSens*FDTSens ; 

disp ('The  overall  probability  of  success  when  the  Retired  Reserve  is  at 

its  highest  is ’ ) 

disp (Ret iredHightotal ) 

Ret iredSensdif f =abs (Ret iredHightot al -Ret iredLowtotal ) ; % computes  the 
difference  between  the  two  extremes  of  the  Retired  Reserve 
disp ('The  probability  difference  is') 
disp (Ret iredSensdif f ) 

'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o 

%These  next  blocks  determine  the  effect  that  the  Force  Protection  has 
SurgReq=most_req; 

ReadyRes=most_redres ; 

Ret iredRes=most_retres ; 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled®!; 

else 

f illed= (ReadyRes+Ret iredRes ) /SurgReq; 

end 
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ForceProt=l ; 


GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) *f illed; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB  (1) +UFTB (2) *For cePr ot +UFTB ( 3 ) *f illed; 

GFTSens=GFTB (1) +GFTB (2) *For ceProt +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For ceProt +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For ceProt +FDTB ( 3 ) *f illed; 
FPCONLow=GenMissionSens* ( ( Acquire_mat *ObtainMatSens ) + (1- 
Acquire_mat ) ) *UFTSens*GFTSens*MFTSens*FDTSens ;  %Finds  the  overall 
probability  based  on  FPCON  Alpha 

disp ('The  overall  probability  of  success  when  the  FPCON  is  Alpha  is') 
disp ( FPCONLow ) 

'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o 

SurgReq=most_req; %set s  surge  required  to  the  most  it  can  be 
ReadyRes=most_redres ; 

Ret iredRes=most_retres ; 

%calculates  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled=l; 

else 

f illed= (ReadyRes+Ret iredRes ) / SurgReq; 

end 

ForceProt=4 ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) * filled; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB (1) +UFTB (2) *For ceProt +UFTB ( 3 ) *f illed; 

GFTSens=GFTB (1) +GFTB (2) *For ceProt +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For ceProt +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) * For ceProt +FDTB ( 3 ) *f illed; 
FPCONHigh=GenMissionSens* ( ( Acquire_mat *ObtainMatSens ) + (1- 
Acquire_mat ) ) *UFTSens *GFTSens *MFTSens *FDTSens ;  %finds  overall 
probability  based  on  the  highest  surge  value 

disp ('The  overall  probability  of  success  when  the  FPCON  is  Delta  is') 
disp (FPCONHigh) 

FPCONdif f =abs (FPCONLow-FPCONHigh) ; %computes  the  difference  between  the 

two  extremes  of  the  FPCON 

disp ('The  probability  difference  is') 

disp (FPCONdiff ) 

'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O 

%The  next  two  blocks  turn  "on"  and  "off"  the  obtain  Material  node  to 
determine 

%the  effect  of  having  it  on  and  off 
SurgReq=most_req; 

ReadyRes=most_redres ; 

Ret iredRes=most_retres ; 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
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f illed=l ; 


else 

f illed= (ReadyRes+Ret iredRes ) / SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) * filled; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB ( 1 ) +UFTB (2) *For ceProt +UFTB ( 3 ) ^filled; 

GFTSens=GFTB  ( 1 )  +GFTB  (2)  *  F  or  ceProt +  GFTB  (  3  )  *f  Hied; 

MFTSens=MFTB ( 1 ) +MFTB (2) *For ceProt +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) * For ceProt +FDTB ( 3 ) ^filled; 

AcqMat total=GenMissionSens*ObtainMat Sens *UFTSens*GFTSens*MFTSens*FDTSen 
s;%finds  overall  probability  based  on  needing  to  accomplish  the  obtain 
Material  node 

disp ('The  overall  probability  of  success  when  it  is  required  to  Acquire 
Material  is ' ) 
disp ( AcqMattotal ) 

'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o'o 

SurgReq=most_req; 

ReadyRes=most_redres ; 

Ret iredRes=most_retres ; 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 
filled=l; 

else 

f illed= (ReadyRes+Ret iredRes ) /SurgReq; 

end 

ForceProt=l ; 

GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) * filled; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
UFTSens=UFTB (1) +UFTB (2) *For ceProt +UFTB ( 3 ) ^filled; 

GFTSens=GFTB ( 1 ) +GFTB (2) *For ceProt +GFTB ( 3 ) *f illed; 

MFTSens=MFTB (1) +MFTB (2) *For ceProt +MFTB ( 3 ) *f illed; 

FDTSens=FDTB (1) +FDTB (2) *For ceProt +FDTB ( 3 ) ^filled; 

Not AcqMat total=GenMissionSens*UFTSens*GFTSens*MFTSens*FDTSens ; % finds 
overall  probability  based  on  not  needing  to  accomplish  the  obtain 
Material  node 

disp (’The  overall  probability  of  success  when  not  required  to  Acquire 

Material  is ’ ) 

disp (NotAcqMattotal ) 

AcqMat sensdif f =abs (NotAcqMattotal-AcqMattotal ) ;%computes  the  difference 
between  needing  to  obtain  materials  and  not  needing  to 
disp (’The  probability  difference  is’) 
disp (AcqMat sensdif f ) 

'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O 
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SurgReq=most_req; 

ReadyRes=most_redres  ; 

Ret iredRes=most_retres ; 

%Finds  the  percent  surge  filled 
if  ( SurgReq<= (ReadyRes+Ret iredRes ) ) 

filled*!; 

else 

f illed= (ReadyRes+Ret iredRes ) /SurgReq; 

end 

ForceProt=l ; 

%The  following  block  finds  the  overall  probability  of  success  when  each 
node  takes 

%on  its  mode  value  and  the  overall  probability  of  success  when  each 
node  takes  on 

%the  value  half  way  between  its  mode  and  1 .  Only  one  node  is  moved  at 
a 

%time  to  see  its  affect  on  the  overall  probability  of  success. 
GenMissionSens=GenMissB ( 1 ) +GenMissB ( 2 ) *ForceProt+GenMissB ( 3 ) *f illed; 
GenMissionSensHigh= ( ( 1-GenMissionSens ) / 2) +GenMissionSens ; 
ObtainMatSens=ob_MatB ( 1 ) +ob_MatB ( 2 ) *ForceProt+ob_MatB ( 3 ) *f illed; 
ObtainMatSensHigh= ( ( 1-ObtainMatSens ) / 2)  +ObtainMatSens ; 

UFTSens=UFTB ( 1 ) +UFTB (2) *For cePr ot +UFTB ( 3 ) *f illed; 

UFTSensHigh= ( (1-UFTSens) / 2) +UFTSens; 

GFTSens=GFTB (1) +GFTB (2) *For ceProt +GFTB ( 3 ) *f illed; 

GFTSensHigh= ( ( 1-GFTSens ) / 2) +GFTSens ; 

MFTSens=MFTB (1) +MFTB (2) * F or ceProt +MFTB ( 3 ) *f illed; 

MFTSensHigh= ( ( 1-MFTSens ) / 2) +MFTSens ; 

FDTSens=FDTB (1) +FDTB (2) *For ceProt +FDTB ( 3 ) *f illed; 

FDTSensHigh= ( ( 1-FDTSens ) / 2) +FDTSens ; 


%This  block  finds  the  difference  between  the  two  probability  of  success 
found  for  each  node  above  to  determine  the  impact  each  node  has  on 
%the  overall  probability  of  success. 

GenMissiondif f = ( (Acquire_mat +ObtainMatSens ) + ( 1- 

Acquire_mat) ) *UFTSens*GFTSens*MFTSens*FDTSens* ( GenMissionSensHigh- 
GenMissionSens ) ; 

ObtainMatdif f =GenMiss ionSens+UF T Sens *GFT Sens +MFT Sens *FDT Sens * ( ( Acquire_ 
mat *ObtainMatSensHigh+ ( 1-Acquire_mat ) ) - ( Acquire_mat *ObtainMatSens+ ( 1- 
Acquire_mat) ) ) ; 

UFTdif f = ( ( Acquire_mat *ObtainMat Sens ) + ( 1- 

Acquire_mat) ) +GFTSens+MFTSens+FDTSens+GenMissionSens+ (UFTSensHigh- 
UFTSens ) ; 

GFTdif f = ( ( Acquire_mat *ObtainMat Sens ) + ( 1- 

Acquire_mat) ) +MFTSens+FDTSens+GenMissionSens+UFTSens+ ( GFTSensHigh- 
GFTSens ) ; 

MFTdif f = ( ( Acquire_mat *ObtainMat Sens ) + ( 1- 

Acquire_mat) ) +FDTSens+GenMissionSens+UFTSens+GFTSens+ (MFTSensHigh- 
MFTSens ) ; 
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FDTdif f = ( ( Acquire_mat *ObtainMat Sens ) + ( 1- 

Acquire_mat) ) *GenMissionSens *UFTSens*GFTSens*MFTSens* (FDTSensHigh- 
FDTSens ) ; 


disp ('The  probability  difference  for  Generate  the  Mission  is’) 
disp ( GenMissiondif f ) 

disp('The  probability  difference  for  Obtain  Materials  is’) 
disp (ObtainMatdif f ) 

disp (’The  probability  difference  for  Undergraduate  Flying  Training  is’ 
disp (UFTdiff ) 

disp ('The  probability  difference  for  Graduate  Flying  Training  is') 
disp ( GFTdif f ) 

disp ('The  probability  difference  for  Mission  Flying  Training  is') 
disp (MFTdif f ) 

disp ('The  probability  difference  for  Flying  Deployment  Training  is') 
disp (FDTdif f ) 


%Displays  the  results  to  a  bar  graph  so  the  user  can  see  which  node  is 
most 

%controlling . 

sensmatrix= [ SurgeSensdif f , ReadySensdif f , Ret iredSensdif f , FPCONdif f , AcqMa 
tsensdif f , GenMissiondif f , ObtainMatdif f , UFTdiff, GFTdif f , MFTdif f , FDTdif f ] 


x«;[l,  2, 3, 4, 5, 6, 7, 8, 9, 10, 11]; 
bar (x,  sensmatrix) 
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Appendix  C.  Example  Data  Input 


Identify  Surge  Requirements 

Given  the  scenario  what  is  the  lowest  likely  number  of  surge 
personnel  required? 

20 

Given  the  scenario  what  is  the  most  likely  number  of  surge 
personnel  required? 

60 

Given  the  scenario  what  is  the  largest  likely  number  of  surge 
personnel  required? 

80 

Recall  Ready  Reserve 

Given  the  scenario,  what  is  the  lower  limit  on  the  expected 
number  of  Ready  Reserve  the  can  be  recalled? 

0 

Given  the  scenario,  what  is  the  expected  number  of  Ready 
Reserve  the  can  be  recalled? 

30 

Given  the  scenario,  what  is  the  upper  limit  on  the  expected 
number  of  Ready  Reserve  the  can  be  recalled? 

40 

Recall  Standby/Retired  Reserve 

Given  the  scenario,  what  is  the  lower  limit  on  the  expected 
number  of  Standby/Retired  Reserve  the  can  be  recalled? 

0 

Given  the  scenario,  what  is  the  expected  number  of 
Standby/Retired  Reserve  the  can  be  recalled? 

2 

Given  the  scenario,  what  is  the  upper  limit  on  the  expected 
number  of  Standby/Retired  Reserve  the  can  be  recalled? 

5 

Force  Protection  Condition 

Given  the  scenario  what  is  the  probability  the  base  is  at  FPCON 

ALPHA(l)? 

0.40 

Given  the  scenario  what  is  the  probability  the  base  is  at  FPCON 

BRAVO(2)? 

0.30 
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_ Acquire  Materiel? _ 

Given  the  scenario,  how  likely  is  it  that  additional  training 
_ materials  will  be  required? _ 


Generate  Training  Mission 

What  is  the  probability  of  Successfully  completing  'Generate  Training 
Mission'  based  on  the  FPCON  and  %  surge  filled  provided? 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.96 

0.03 

ALPHA(l) 

0 

0.65 

DELTA(4) 

0 

0.7 

Obtain  Training  Material 

What  is  the  probability  of  Successfully  completing  'Obtain  Training 
Material'  based  on  the  FPCON  and  %  surge  filled  provided? 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

i 

0.03 

ALPHA(l) 

0 

0.85 

DELTA(4) 

0 

0.6 

Undergraduate  Flying  Training 

What  is  the  probability  of  Successfully  completing  'Undergraduate 
Flying  Training'  based  on  the  FPCON  and  %  surge  filled  provided? 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.98 

0.03 

ALPHA(l) 

0 

0.6 

DELTA(4) 

0 

0.5 
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Graduate  Flying  Training 


What  is  the  probability  of  Successfully  completing  'Graduate  Flying 
Training'  based  on  the  FPCON  and  %  surge  filled  provided? 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.98 

0.03 

ALPHA(l) 

0 

0.6 

DELTA(4) 

0 

0.5 

Mission  Flying  Training 

What  is  the  probability  of  Successfully  completing  ’Mission  Flying 

1  Training’  based  on  the  FPCON  and  % 

surge  filled  provided?  | 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.99 

0.03 

ALPHA(l) 

0 

0.7 

DELTA(4) 

0 

0.6 

Flying  Deployment  Training 

What  is  the  probability  of  Successfully  completing  ’Flying  Deployment 

Training'  based  on  the  FPCON  and  % 

surge  filled  provided? 

FPCON 

%  surge  filled 

P(Success) 

ALPHA(l) 

100 

0.99 

0.03 

ALPHA(l) 

0 

0.75 

DELTA(4) 

0 

0.65 

106 


Appendix  D.  Acronyms 


ACS  Agile  Combat  Support 
AcV  Acquisition  Viewpoint 
AD  Active  Duty 
AF  Air  Force 
AFI  Air  Force  Instruction 
AFPD  Air  Force  Policy  Directive 
ARC  Air  Reserve  Component 
AV  All  View(point) 

C4ISR  Command,  Control,  Communication,  Computers,  Intelligence,  Surveillance, 
Reconnaissance 

CADM  Core  Architecture  Data  Model 

CBA  Capability  Based  Assessment 

CBAM  Capability  Based  Assessment  Methodology 

CBP  Capability  Based  Planning 

CCA  Clinger-Cohen  Act 

CJCS  Chairman  of  the  Joint  Chiefs  of  Staff 

COCOM  Combatant  Commander 

CONOPS  Concept  of  Operations 

CRRA  Capability  Review  and  Risk  Assessment 

CSAF  Chief  of  Staff  of  the  U.S.  Air  Force 

CV  Capability  Viewpoint 
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DIV  Data  and  Information  Viewpoint 
DM2  DoDAF  Meta-Model 
DoD  Department  of  Defense 

DoDAF  Department  of  Defense  Architecture  Framework 

DOTMLPF  Doctrine,  Organization,  Training,  Materiel,  Leadership  and  Education, 
Personnel,  and  Facilities 
FAA  Functional  Area  Analysis 
FDT  Flying  Deployment  Training 
FFBD  Functional  Flow  Block  Diagram 
FNA  Functional  Needs  Analysis 
FSA  Functional  Solutions  Analysis 
GE  Good  Enough  Value 
GFT  Graduate  Flying  Training 
ICOM  Input  Control  Output  Mechanism 
IEEE  Institute  of  Electrical  and  Electronics  Engineers 
IPL  Integrated  Priority  List 
ISO  International  Standards  Organization 
JCA  Joint  Capability  Area 

JCIDS  Joint  Capabilities  Integration  and  Development  System 

JIC  Joint  Integrating  Concept 

JOC  Joint  Operating  Concept 

JROC  Joint  Requirements  Oversight  Council 

LMV  Limited  Military  Value 
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MAJCOM  Major  Command 
MCL  Master  Capability  Library 
MFT  Mission  Flying  Training 

MODAF  Ministry  of  Defense  Architecture  Framework 
OSD  Office  of  the  Secretary  of  Defense 
OV  Operational  View(point) 

PAAR  Potential  Area  to  Accept  Risk 

POM  Program  Objective  Memorandum 

PPBE  Planning  Programming  Budgeting  and  Execution 

PSM  Process  Sequence  Model 

PV  Project  Viewpoint 

RGS  Requirements  Generation  System 

SME  Subject  Matter  Expert 

SMP  Strategic  Master  Plan 

SOV  Service  Oriented  Viewpoint 

StdV  Standards  Viewpoint 

StV  Strategic  Viewpoint 

SvcV  Service  Viewpoint 

SV  Systems  View(point) 

TOGAF  The  Open  Group  Architecture  Framework 
TV  Technical  View 
UFT  Undergraduate  Flying  Training 
UML  Unified  Modeling  Language 
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